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Preface 

Pattern  recognition  using  self-organizing  machines 
was  chosen  as  a  thesis  topic  because  of  my  rapidly  increas¬ 
ing  interest  in  electronic  computers.  The  subject  was  aug- 
gssted  by  Captain  Prank  H.  Brown.  He  had  found  that  no 
investigation  and  comparison  of  the  pattern  recognition 
capabilities  of  three  classes  of  self -organising  machines 
existed  in  the  literature. 

It  was  first  believed  that  a  mathematical  study  in  the 
form  of  theorems  and  proofs  could  be  made.  This  idea  was 
soon  discarded  because  it  was  far  too  oomplex.  A  period  of 
thrashing  followed  during  which  attempts  were  made  to  find 
a  method  of  attacking  the  problem.  It  was  finally  dscided 
to  work  through  a  large  number  of  pattern  recognition  pro¬ 
blems  using  mathematical  models  of  typical  machines  from 
each  of  the  three  olasses. 

Because  each  problem  required  a  very  large  number  of 
simple  calculations  and  comparisons,  computer  programs  were 
written  to  simulate  the  machines.  Mathematical  models 
suitable  for  computer  programming  had  to  be  derived  from 
the  somewhat  oonfusing  literature  available.  Vhlle  the 
mathematical  models  were  being  developed,  a  more  thorough 
understanding  of  some  of  the  basio  pattern  recognition 
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techniques  was  acquired. 

It  soon  beoame  evident  that  the  maohines  being  studied 
were  not  (as  Voltaire's  Candid®  would  say)  the  best  possible 
maohines  in  this  best  of  all  possible  worlde.  Therefore, 
an  attempt  was  made  to  design  a  simpler  and  yet  more  capable 
self-organizing  pattern  recognition  unit.  The  SOPRU  of 
Chapter  IV  is  a  direct  result  of  this  attempt. 

It  is  my  personal  opinion,  formed  during  the  previously 
described  investigation,  that  the  correlation  approaoh  is 
the  most  promising  solution  to  the  pattern  recognition  pro¬ 
blem.  The  correlation  technique  can  be  improved  still 
further  by  combining  it  with  the  adjustable  weight  threshold 
logie  approach. 

It  is  a  pleasure  to  acknowledge  the  guidance  and  help 
given  to  me  by  the  following  people:  Captain  Frank  U.  Brown, 
my  faculty  thesis  advisor,  for  patiently  providing  guidance 
for  sty  efforts;  Mr.  Ceoil  V.  Gwinn  of  the  Bionics  section 
of  the  Avionics  Laboratory  of  Wright-Patterson  Air  Force 
Base  for  providing  reports  and  information;  and  Miss 
Jaoqueline  Collins  for  cheerfully  and  accurately  trans¬ 
lating  my  illegible  manuscript  into  typewritten  form. 

Ailing  C.  Foreman 
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Abstract 

Most  of  the  pattern-recognition  self-organizing  ma¬ 
chines  can  be  classified  as  adjustable-weight  threshold- 
logic  machine#,  statistical-switching  machines,  or  corre¬ 
lation  machines.  A  noisy  pattern  is  a  pattern  that  Taries 
slightly  from  a  model  pattern  that  the  machine  has  been 
taught.  Computer  simulations  of  a  typioal  machine  from 
each  of  the  three  classes  indicate  that  the  noisy  pattern 
recognition  capabilities  are  poor  for  statistical-switching 
machines,  good  for  threshold-logic  machines,  best  for 
correlation  type  maohines.  A  proposed  correlation  self¬ 
organizing  maohine  is  simple,  learns  in  one  step,  and 
recognizes  noisy  patterns  accurately. 
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I.  Introduction 

One  of  man'e  greatest  aaaeta  ia  hie  ability  to  obtain 
information  from  hia  environment  and  to  make  intelligent 
deoiaiona  concerning  thla  information.  The  information  is 
received  from  the  environment  through  sense  organs  such  as 
the  eyes  and  ears.  The  information  is  then  sent  to  the 
brain  where  it  must  be  classified  before  a  decision  can  be 
made  to  respond  to  the  information,  or  stimulus,  in  a  par¬ 
ticular  manner.  This  classification  process  is  called 
pattern  recognition. 

A  basic  limitation  of  electronic  computers  is  their 
ability  to  gain  information  from  their  environment.  All  in¬ 
formation  must  be  coded  and  given  to  the  machine  in  a  very 
special  format.  This  coding  process  is  very  difficult  for 
humans  because  any  very  slight  coding  error  may  give  the 
computer  completely  false  Information.  A  possible  way  to 
overcome  this  language  barrier  is  to  give  the  machine  some 
pattern  recognition  capabilities. 

A  computer  with  the  ability  to  recognize  patterns  could 
be  used  to  perform  some  of  the  more  tedious  tasks  now  done 
by  humans.  Thus,  a  typewritten  page  could  be  typed  from  a 
manuscript  by  a  typewriter  controlled  by  a  pattern  recogni¬ 
tion  maohlne.  A  machine  that  would  be  of  more  interest  to 
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the  Air  Force  and  Navy  would  he  able  to  spot  enemy  planes 
on  a  radar  scope  or  submarines  by  the  return  pattern  of  a 
sonar  signal.  Suoh  a  machine  could  have  a  higher  degree  of 
reliability  than  humans  because  the  maohine  would  not  get 
bored  as  a  human  does. 

Classical  logical  design  procedures  using  standard 
AND,  OH,  and  NOT  components  have  been  used  in  some  cases  to 
develop  pattern  recognition  devices.  These  devioes  are  not 
flexible  enough  to  solve  general  pattern  recognition  pro¬ 
blems.  Any  slight  variation  in  the  pattern  requires  a 
completely  new  design. 

A  more  flexible  way  to  construct  a  pattern  recognition 
device  is  to  use  fixed  weight  threshold  logic.  Slight  var¬ 
iations  in  the  pattern  will  not  have  a  tendency  to  cause  a 
change  in  the  output  of  a  threshold  gate.  The  values  of 
the  weights  may  be  determined  by  using  a  digital  computer 
(Ref  7). 

The  weight  determination  procedure  can  be  very  compli¬ 
cated.  A  large  change  in  the  patterns  will  require  a  com¬ 
pletely  new  set  of  weights.  Thus,  the  fixed-weight  thres¬ 
hold  logic  approach  to  the  pattern  recognition  problem  Is 
also  somewhat  inflexible. 

Perhaps  the  best  method  is  to  use  self -organizing 
machines  to  solve  pattern  recognition  problems.  A  self¬ 
organizing  maohine  is  sometimes  oalled  a  learning  machine 
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because  it  has  the  ability  to  adjust  some  parameter  of  its 
circuit  to  produce  a  desired  output  for  any  possible  input. 
Thus,  self-organizing  machines  are  ezoellent  for  solving 
the  complioated  design  problems  of  pattern  recognition 
units. 

At  least  three  classes  of  self -organizing  machines  have 
been  proposed  in  the  last  few  years.  These  classes  are: 

1.  Adjustable-weight  threshold-logic  machines. 

2.  Statistioal-Switching  Machines. 

3.  Correlation  Machines. 

Each  of  these  classes  of  machines  has  certain  advantages  and 
di sadvant age s . 

Object 

The  object  of  this  thesis  is  to  investigate  and  compare 
the  pattern  learning  and  recognition  capabilities  of  each  of 
the  three  classes  of  self-organizing  machines.  Particular 
emphasis  will  be  placed  on  the  abilities  of  the  machines  to 
recognize  patterns  that  are  slightly  different  from  the 
model  patterns  that  the  machines  have  been  taught  to  recog¬ 
nize. 

Soope 

The  analysis  will  be  restricted  to  oombinatorial-logic 
machines.  Thus,  the  recognition  of  sequential  patterns 
will  not  be  considered. 
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The  sensor  elements  and  the  praotioal  problems  of  pre¬ 
senting  the  patterns  to  the  machine  will  not  be  disoussed. 
It  will  be  assumed  that  all  sensor  units  perform  correct 
measurements  on  the  pattern,  and  that  the  outputs  of  the 
sensor  units  contain  the  information  necessary  to  specify 
the  value  of  the  measurement. 


Notation 

The  symbols  used  in  this  report  are: 


AND  GATE 

OB  Gate 

NOT  Box 

Statistical  Switch 

Threshold  Gate  with  Threshold  6 

Bistable  Multivibrator 

Algebraic  Summation 


+  Logical  Disjunction,  OR 


4 


SE/EE/63-8 

AND  will  be  denoted  by  juxtaposition  of  the  Boolean  varia¬ 
bles.  The  plus  sign  is  used  both  for  algebraio  addition  and 
logical  disjunction,  but  the  meaning  can  be  found  from  the 
context.  Unless  otherwise  specified,  the  weight  values  on  the 
Inputs  of  threshold  gates  are  assumed  to  be  one. 

Organization 

Eaoh  of  the  next  three  ohapters  will  discuss  one  of  the 
classes  of  self-organizing  machines.  Adjustable-weight 
threshold-logic  maohines,  statistical-switching  machines, 
and  correlation  machines  will  be  described  in  Chapters  II, 

III,  and  IV  respectively.  Each  chapter  contains  a  mathe¬ 
matical  analysis  of  a  machine,  a  discussion  of  a  computer 
simulation  of  that  machine,  and  the  results  of  the  computer 
simulation  problems. 

Chapter  V  consists  of  a  comparison  of  the  results  of 
the  investigation,  conclusions,  and  recommendations  for 
further  study. 
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II.  Adjustable-Weight  Threahold-Loglo  Machines 

The  first  self-organizing  maohines  to  be  proposed  for 
use  in  solving  the  pattern  recognition  problem  belong  to 
the  class  of  learning  machines  that  use  threshold  gates 
with  adjustable  weights.  A  single-stage  devloe  proposed 
by  Mattson  (Ref  8),  and  CHIU)  (Ref  4),  an  analog-input  ma- 
ohine  being  built  for  the  Air  Force,  will  be  very  briefly 
described  in  this  ohapter.  A  more  thorough  description  of 
a  more  general  machine,  the  peroeptron  (Ref  9 ,  10,  11), 
will  then  be  given.  Finally,  a  computer  simulation  design¬ 
ed  to  determine  experimentally  some  of  the  pattern  recog¬ 
nition  capabilities  of  a  simple  peroeptron  will  be  discus¬ 
sed.  Before  beginning  the  description  of  the  maohines,  an 
explanation  of  some  of  the  properties  of  threshold  logic 
is  in  order. 

Threshold  Logic 

Threshold  logic  concerns  a  special  class  of  Boolean 
switching  functions.  A  threshold  function  exists  if  a  set 
of  n  input  weights  w1 ,  Wg*  •  •  .w^, .  •  ,wa  and  a  threshold  value 
t  can  be  found  whioh  satisfy  the  inequalities 

n 

=  t  (la) 

i«1 
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for  all  combinations  of  the  n  Boolean  variables  x1 ,  x2>... 
zl’***zn  *kat  are  logically  true  and 


I 


wixi 


t 


(1b) 


for  all  combinations  of  the  variables  that  are  false.  For 
a  gate  to  be  a  threshold  gate  two  conditions  must  be  met  as 
follows: 

1 .  The  output  of  the  gate  must  be  a  logical  one  if  and 
only  if  the  linear  sum  of  the  weighted  inputs  is  greater 
than  or  equal  to  the  threshold  value. 

2.  The  output  of  the  gate  must  be  a  logical  zero  if 
and  only  if  the  linear  sum  of  the  weighted  inputs  is  less 
than  the  threshold  value. 

An  Important  advantage  of  a  threshold  gate  over  stand¬ 
ard  AND  and  OB  type  gates  is  the  ability  of  the  threshold 
gate  to  generate  many  different  Boolean  functions  by  simply 
varying  the  input  weights  and  threshold  value.  Fourteen  of 
the  16  Boolean  functions  of  two  variables  can  be  realized 
by  a  single  threshold  gate.  Thus,  a  considerable  reduction 
in  the  number  of  oomponents  required  to  construct  logic 

oircuits  can  be  made  by  using  threshold  loglo.  Unfortunate- 

2n 

ly,  the  fraction  of  the  2  Boolean  funotions  of  n  varia¬ 
bles  that  are  threshold  functions  deoreases  rapidly  as  n 


7 


GE/EE/63-8 

increases.  However,  since  any  Boolean  function  can  be  syn¬ 
thesized  using  AND,  OH,  and  NOT  gates,  and  sinoe  a  thres¬ 
hold  gate  can  generate  these  functions,  any  Boolean  logical 
device  can  be  constructed  using  threshold  gates. 

Threshold  functions  are  sometimes  called  linearly 
separable  functions.  The  reason  for  this  name  can  be  seen 
if  the  2n  minterms  of  n  variables  are  represented  as  ver¬ 
tices  of  an  n-dimen8ional  cube.  If  the  vertices  corres¬ 
ponding  to  true  minterms  can  be  separated  from  the  vertices 
corresponding  to  false  minterms  by  an  n-1  dimensional  hyper¬ 
plane,  then  the  function  is  a  threshold  function. 

Example 

Given  the  three  variable  Boolean  functions: 

?1  -  X1X2X3  +  X1X2X3  <2) 

J?2  -  +  x1x2x,  (3) 

These  equations  are  represented  by  the  solid  black  vertices 
of  the  cubes  shown  below. 
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lotiee  that  Eq  (2)  is  a  threshold  function  since  a  2-dimen- 
sional  plane  oan  separate  the  solid  hlaok  vertices  from  the 
other  vertices ,  hut  Eq  (3)  is  not  a  threshold  function 
since  no  plane  can  separate  the  solid  black  vertices  from 
the  rest  of  the  vertices. 

If  Ineqs  ( 1 )  are  reduced  to  an  equation,  the  resulting 
equation  will  define  the  n-1  dimensional  hyperplane  that 
separates  the  vertices  representing  true  minterms  from  the 
vertices  representing  false  minterms.  The  input  weights 
are  the  direction  numbers  of  the  plane  and  the  threshold 
t  is  the  squared  perpendicular  distance  from  the  plane  to 
the  origin.  The  plane  required  to  separate  the  vertices 
is  not  unique;  therefore,  many  different  combinations  of 
weights  and  threshold  values  can  be  used  to  synthesize  one 
Boolean  logic  network. 

The  Hamming  distance  between  two  minterms  is  defined 
as  the  number  of  edges  of  the  n-dimensional  cube  that  must 
be  traveled  to  get  from  the  vertex  corresponding  to  one 
minterm  to  the  vertex  corresponding  to  the  other  minterm. 

It  can  be  seen  from  the  figure  of  the  previous  example  that 
only  one  variable  is  changed  in  going  from  one  vertex  to 
the  next.  Thus  Hamming  distance  is  equal  to  the  number  of 
corresponding  variables  that  are  different  in  two  minterms. 

A  charaoteristlc  of  threshold  functions  is  that  min¬ 
terms  that  are  close  to  true  minterms  tend  to  be  classified 
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as  true  mlnterms  and  mlnterms  that  are  olose  to  false  min- 
tsrms  tend  to  be  olassified  as  false  mlnterms.  This  is  a 
result  of  the  linear  separability  property.  All  mlnterms 
with  vertices  on  one  side  of  the  plane  will  be  classified 
as  true  and  all  mlnterms  with  corresponding  vertices  on  the 
other  side  of  the  plane  will  be  olassified  as  false.  Ver¬ 
tices  far  away  from  the  plane  will  be  with  all  of  their 
olose  neighbors  in  the  same  classification.  Vertices  close 
to  the  plane  will  have  neighbors  in  both  classifications. 

Because  a  threshold  gate  will  tend  to  give  the  same 
response  for  mlnterms  that  are  close  to  each  other,  thres¬ 
hold  logic  can  be  used  to  build  pattern  recognition  devices 
that  are  relatively  insensitive  to  slight  variations  in  the 
pattern.  Furthermore,  the  device  can  be  constructed  so 
that  it  will  adjust  the  weights  on  the  inputs  of  the  thres¬ 
hold  gates  in  such  a  manner  that  it  will  learn  to  recognize 
the  patterns  that  are  presented  to  it.  This  is  the  basis 
for  the  maohines  that  will  be  discussed  in  the  remaining 
portion  of  this  chapter. 

Mattson's  Machine 

A  self-organizing  binary  device  proposed  by  Mattson 
(Ref  8)  is  shown  in  Fig.  1.  It  consists  of  a  threshold 
gate  with  continuously  variable  weights  on  the  inputs  and 
a  continuously  variable  threshold  value.  A  training  unit 
samples  the  performance  of  the  system  and  determines  what 
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corrections  to  make  to  the  weights  and  threshold  value. 

Since  the  device  can  only  separate  minterme  that  are  lin¬ 
early  separable,  there  is  a  very  limited  number  of  appli¬ 
cations  for  a  single  stage  of  these  devices.  A  multi-stage 
network  of  these  learning  machines  can  be  used  to  realize 
a  larger  class  of  Boolean  functions,  however. 

CHIU) 

A  Cognitive  Hybrid  Intelligent  Learning  Device  (CHILD), 
designed  by  Choisser  (Ref  4)  is  presently  being  constructed 
and  studied  at  the  Rome  Air  Development  Center  in  Hew  York. 
The  inputs  to  a  CHILD  are  in  the  form  of  an  analog  vector 
with  n  terms.  Each  input  is  fed  to  a  CHILD  cell  (Pig.  2) 
which  has  an  output  other  than  zero  if  and  only  if  the  value 
of  the  input  is  between  two  threshold  values  as  shown  in 
the  transfer  characteristic  shown  below. 

Output 

- I - 1 


®u  Input 

The  outputs  of  each  cell  are  weighted  and  fed  to  the  input 
of  a  threshold  gate.  The  output  of  the  threshold  gate  is 
a  logioal  one  if  and  only  if  the  sum  of  the  outputs  of  the 
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An  Adjustable  Linary  Logical  Network 
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Pig.  2 

A  CHILD  Network  for  One  Pattern  Class 
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CHILD  oells  is  greater  than  a  threshold  value. 

Eaoh  olass  of  patterns  to  be  reoognized  must  be  allot¬ 
ted  one  threshold  gate  and  n  CHILD  oells.  A  training  de¬ 
vice  will  adjust  the  upper  and  lower  threshold  values  on 
the  inputs  of  the  oells  and  the  weights  on  the  outputs  of 
the  oells  until  the  device  has  learned  to  respond  to  the 
inputs  in  the  desired  manner. 

T&e  Perception 

The  peroeptron  (Hef  9,  10,  11)  conceived  by  Rosenblatt, 
has  been  the  object  of  research  sponsored  by  the  Navy  at 
Cornell  Aeronautical  Laboratory.  A  simple  perceptron  has 
two  stages  of  threshold  logic.  Sensor  elements  are  con¬ 
nected  to  the  first  stage  threshold  gates  by  leads  with 
weights  chosen  from  the  set  £0,  1,  -1  )  in  a  random  manner. 
The  first  stage  outputs  are  weighted  and  fed  to  the  second 
stage  threshold  gates.  These  seoond  stage  weights  may  be 
adjusted  so  that  the  seoond  stage  threshold  gates  will  have 
a  logioal  one  output  for  some  input  combinations  and  a  log¬ 
ical  zero  output  for  other  combinations. 

Mathematical  Analysis.  A  pattern  presented  to  a  per¬ 
oeptron  (Pig.  3)  will  cause  the  sensor  (S)  units  to  have 
a  zero  output  if  the  sensor  unit  is  not  excited  and  a  one 
output  if  the  S  unit  is  exoited.  The  outputs  of  the  sensor 
units  can  be  arranged  in  the  form  of  an  n-element  binary 
vector 
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SENSOR  ASSOCIATION  RESPONSE 

UNITS  UNITS  UNIT 


Pig.  3 

A  Simple  Perceptron 
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S  ■ 


81 

*2 

s3 


(4) 


where  each  s^  can  have  the  value  zero  or  one.  This  vector 
will  often  be  referred  to  as  the  Input  vector,  and  will 
often  be  written  In  the  transposed  form 


s2...Sj 


e  e  • 


(5) 


The  outputs  of  the  sensor  units  are  conneoted  to 
association  (A)  units  with  weighted  oonneotions.  These 
weighted  oonneotions  can  be  arranged  into  a  topology 
matrix 

t12**,tij“*t1n 
t22*,*t2j**,t  2n 

(6) 

•  •  • 

e  e  e 

tm2*,*tmj*#,tmn 

Where  each  t^  is  the  weighted  connection  of  the  jth  S  unit 
to  the  ith  A  unit,  and  m  and  n  are  the  number  of  A  units 
and  S  units,  respectively.  The  value  for  eaoh  t^  is 
ohosen  by  consulting  a  random  number  table.  Thus,  the  matrix 
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irlll  be  composed  of  a  random  array  of  zeros,  ones,  and  minus 
ones. 


The  output  of  each  A  unit  will  be  a  logical  one  If  and 
only  if  the  sum  of  the  weighted  Inputs  is  greater  than  the 
threshold  value,  and  will  be  zero  otherwise.  This  can  be 
expressed  as 


1 


0 


tij  ej  =  ®i 
hi  Bi  *  ei 


(7) 


where  is  the  output  and  9^  is  the  threshold  value  of 
the  ith  A  unit. 


The  outputs  of  the  A  units  are  multiplied  by  the  ad¬ 
justable  weights  and  summed  at  the  input  of  the  response 
(R)  unit.  The  output  of  the  response  unit,  which  is  the 
output  of  the  perceptron,  will  be  a  logical  one  if  the  sum 
of  the  weighted  outputs  of  the  A  units  is  greater  than  or 
equal  to  the  threshold  value,  and  will  be  zero  otherwise. 
Thus, 


R 


*  m 

1  if  Ys  wiai 

i=i 

< 

m 

0  if  £ 

i=1 


0. 


0, 


(8) 
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where  w^  is  the  weight  of  the  connection  of  the  ith  A  unit 
to  the  B  unit,  and  ie  the  threshold  value  of  the  B  unit 
threshold  gate. 

Humber  of  Units  Bequired.  The  number  of  sensor  units 
required  for  a  pattern  recognition  problem  depends  upon 
the  complexity  of  the  patterns.  More  details  of  a  pattern 
can  be  used  to  classify  the  pattern  if  a  higher  number  of 
sensors  is  used.  In  the  Hark  I  perceptron  built  at  Cor¬ 
nell,  400  sensor  unite  were  used  in  a  20  x  20  array  (Bef 
10).  Flans  for  the  Mark  II  perceptron  indicated  that 
approximately  4000  sensor  units  would  be  used. 

The  number  of  association  units  required  for  the  per- 
oeptron  depends  upon  the  number  of  sensor  units  used. 
Ideally,  211  A  units  should  be  used  to  insure  that  the  A 
unit  output  vectors  corresponding  to  the  input  vectors  of 
the  patterns  to  be  learned  are  not  logically  dose  in  terms 
of  Hamming  distance.  Hote  that  no  adjustment  of  the  input 
weights  of  the  B  unit  will  allow  the  perceptron  to  distin¬ 
guish  between  two  input  vectors  with  the  same  A  unit  re¬ 
sponses.  The  number  2n  gets  extremely  large  as  n  in¬ 
creases,  however,  so  a  more  realistic  number  of  A  units 
must  be  used.  The  number  of  A  units  used  in  the  Hark  I 
perceptron  was  only  512,  but  the  number  planned  for  the 
Hark  II  approaohed  100,000. 

The  number  of  B  units  required  depends  upon  the 
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number  of  different  claseificatione  to  be  learned.  One  R 
unit  could  be  used  for  each  claee,  but  thia  would  be  in¬ 
efficient  beoauee  the  output  can  be  encoded  with  a  much 
smaller  number.  In  fact,  the  number  of  R  units  only  needs 
to  be  greater  than  or  equal  to  log2k  where  k  is  the  number 
of  classifications  to  be  made,  for  a  completely  different 
binary  code  for  each  class.  Thus,  six  R  units  are  required 
to  encode  the  alphabet  and  ten  numbers,  sinoe  six  is  equal 
to  log264  which  is  greater  than  log262. 

Organization  Procedure .  Several  training  methods  have 
been  Investigated  for  the  simple  perceptron  at  Cornell. 

The  most  successful  of  these  is  the  cx  -error  correction 
procedure.  In  fact  it  has  been  proven  that  this  procedure 
will  yield  a  solution  to  the  pattern  classification  problem 
in  finite  time,  provided  each  pattern  is  presented  again  in 
finite  time  in  any  sequence,  and  provided  a  solution  exists 
(Ref  11:18).  For  a  solution  to  exist,  the  responses  of  the 
A  units  to  the  various  input  vectors  must  be  linearly  sep¬ 
arable  . 

The  c* -error  correction  system  ohanges  the  weights 
on  the  R-unit  input  connectione  only  if  the  perceptron  re¬ 
sponse  is  not  correct,  and  only  if  the  A  unit  associated 
with  the  weighted  connection  had  a  logical  one  output.  In 
general,  the  value  of  the  change  can  be  variable  and  need 
not  be  an  integer.  For  the  computer  simulations  to  be 
described,  the  value  of  the  weight  change  will  be  a 
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constant  equal  to  ona.  Thus,  the  correction  to  the  ith 
weight  can  he  formulated  as 

("i^new  3  ^wi^old  +  ai^Rdesired  “  Ractual^  ^ 

where  wi  is  the  weight  of  the  connection  of  the  ith  A  unit 
to  the  R  unit,  is  the  output  of  the  ith  A  unit,  Ra98iraa 
is  the  desired  response  of  the  peroeptron  to  the  input 
vector,  and  Bac-fcual  is  the  actual  response  of  the  peroep¬ 
tron. 

It  has  been  shown  that  the  order  of  presentation  of 
the  input  vectors  to  the  perceptron  does  not  affect  the 
ability  of  the  perceptron  to  learn  to  recognise  the  pat¬ 
terns  (Ref  11:18).  It  will  affect  the  learning  time,  how¬ 
ever.  In  the  computer  simulation  described  later  in  the 
chapter,  the  patterns  were  presented  so  that  the  desired 
responses  were  an  alternating  sequence  of  zeros  and  ones. 
After  all  inputs  were  presented,  the  sequence  was  repeated. 

Hoisv  Patterns .  For  a  pattern  recognition  device  to 
be  of  any  "value ,  it  must  be  capable  of  recognizing  a  pat¬ 
tern  even  though  there  is  a  slight  variation  in  the  pattern. 
Slight  variations  of  the  input  vector  from  a  model  vector 
which  the  perceptron  has  learned  to  recognise  will  be 
termed  noise.  Thus,  if  one  digit  of  the  input  vector  is 
different  from  the  corresponding  digit  of  the  model  veotor, 
the  pattern  will  be  said  to  have  one  digit  noisy. 
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Computer  Simulation  of  a  Simple  Perceptron 

It  was  decided  that  a  good  way  to  determine  the  noisy 
pattern  recognition  capabilities  of  a  simple  perceptron 
would  be  to  work  through  several  example  problems  to  see 
how  well  the  perceptron  classified  the  noisy  patterns. 

Since  hand  calculations  of  this  sort  would  be  extremely 
tedious,  several  oomputer  programs  were  written  by  the 
author  to  simulate  the  perceptron.  A  typioal  program  for 
an  IBM  1620  digital  computer  is  discussed  in  Appendix  A. 

The  perceptron  simulated  for  the  first  two  problems 
had  25  sensor  units,  30  association  units,  and  one  response 
unit.  The  number  of  sensor  units  was  reduced  to  nine  for 
the  third  problem.  All  threshold  values  and  the  initial 
values  of  the  weights  were  zero.  Instead  of  consulting  a 
random  number  table  to  obtain  a  random  topology  (  [t] ) 
matrix,  a  short  maohine  language  program  was  written  to 
generate  a  random  array  of  zeros,  ones,  and  minus  ones, 
and  to  punch  the  array  onto  IBM  cards.  Another  machine 
language  program  was  used  to  translate  model  input  vec¬ 
tors  into  noisy  input  vectors  and  to  punoh  the  noisy  vec¬ 
tors  onto  cards. 

The  remaining  portion  of  this  chapter  will  be  used 
to  discuss  three  pattern  recognition  problems  and  the 
results  of  the  problems. 

Problem  2*  The  first  problem  to  be  presented  to 
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the  perceptron  simulation  program  was  to  learn  to  plaoe  the 


letters  A, 

B, 

c, 

D, 

and 

E 

into 

one  classification,  and 

the 

numbers  1 , 

2, 

3» 

4, 

and 

5 

into 

another  classification. 

The 

desired  response  for  the  letters  was  arbitrarily  chosen  to 
he  a  logioal  one,  and  the  desired  response  for  the  numbers 
was  chosen  to  be  a  logical  zero.  The  model  letters  and 
numbers  are  shown  on  the  3x5  grids  of  Fig.  4.  The  dark 
squares  correspond  to  sensor  units  that  have  a  logical  one 
output,  and  the  light  squares  correspond  to  sensor  units 
that  have  logical  zero  outputs.  The  input  vectors  were 
coded  from  the  patterns  by  scanning  each  row  from  left  to 
right,  starting  with  the  top  row,  and  placing  a  zero  in  the 
input  vector  for  each  light  square  and  a  one  for  each  dark 
square.  The  numbers  in  the  grid  below  are  the  positions  of 
the  elements  of  the  grid  in  the  input  veotors. 


1 

2 

3 

n 

5 

6 

D 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

The  model  input  vectors  for  the  letters  of  Fig.  4  are  shown 
below  in  the  order  that  they  were  presented  to  the  per¬ 
ceptron. 
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sj  *0010001010010101  1  1  11  10001 

S?  -  0010000100001000010000100 

§2=1110010010111001001011100 

s!  =  0  1  1  1  0  1  0  0  0  1  0  0  1  1  0  0  1  0  0  0  1  1  111 

sj  »  0  1  11010000100001000001  110 
0  (10 

si  =1111100010001101000101110 

"3 

§£  =  1  1  10010010100101001011  100 

sj  *  0  0  1  10010101  1  1  1  10001000010 

§2=1  1  1  101  00001  1  1  00100001  1  1  10 

si  =1111010000111100000101110 
*  0 

After  the  perceptron  had  learned  to  correctly  classify 
the  model  input  rectors,  500  samples  of  noisy  input  rectors 
were  presented  to  the  computer  to  find  out  if  they  would 
be  classified  correotly.  No  weight  adjustments  were  made 
during  these  tests.  Each  of  the  digits  of  each  of  the 
model  input  rectors  was  ohanged  only  once  to  generate  250 
input  rectors  with  one  digit  noisy.  Two  digits  were  per¬ 
turbed  in  each  of  the  last  250  noisy  input  rectors. 

Results  of  Problem  U  The  perceptron  learned  to  prop¬ 
erly  olassify  each  of  the  10  model  rectors  after  each  rec¬ 
tor  had  been  presented  to  the  machine  only  twice. 

The  peroeptron  made  22  mistakes  in  classifying  the 
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Character 

Number  of 
Mistakes 

Recognition 

Level 

Critical 

Digits 

A 

5 

a6%  ! 

■MWHEfWJKilK*: 

B 

mmm. ismtm 

C 

3 

m _ 

2,  7,  1b  ~ 

D 

6 

100* 

E 

2 

20.  25 

i 

0 

100* 

2 

0 

r  too* ^ 

3 

2 

13.  24 

4 

7_ i 

72* 

1 ,  4 ,  5 ,  TO ,  14,  20 , 

24 

5 

■■■■■■I 
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Table  I 

Results  of  One-Noisy  Digit  Tests  of  Problem  1 


Character 

Number  of 
Mistakes 

Recognition 

Level 

2  Critical 

Digits 

A 

7 

72* 

TIMMUKM 

"  B' . 

3 

—  5 5* 

KKiOBMOKIWI 

0 

hhhhi 

■BH 

2,  16  1 

mmmmm 

0 

E 

6 

76* 

■ 

■CTMHKBKBiM 

1 

0 

100* 

2 

0 

100* 

3 

5 

80* 

ramisKKm 

wjkhbmHM 

4 

1 1 

56* 

2,  10;  5,  17;  6,  18; 
7,  21;  10,  24;  14, 

20;  15,  18;  18,  2;  20 
5;  21,  6;  24,  25 _ 

5 

4 

l 

84* 

EKtMHHI 

Table  II 


Results  of  Two-Noisy  Digit  Tests  of  Problem  1 
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250  vectors  with  one  noisy  digit,  so  the  correct  response 
was  given  for  91.33*  of  the  samples.  Table  1  shows  the  num¬ 
ber  of  mistakes  made  in  classifying  each  letter  fluid  number, 
and  which  digits  were  noisy  when  the  mistakes  were  made. 

Thirty-seven  of  the  250  input  vectors  with  two  digits 
noisy  were  not  classified  correctly.  Thus,  the  overall 
recognition  level  was  853*  for  this  case.  A  listing  of  the 
mistakes  is  shown  in  Table  2. 

Problem  2.  For  the  second  problem  the  number  of  pat¬ 
tern  classes  was  increased  to  25.  The  perceptron  was  re¬ 
quired  to  learn  to  classify  the  Arabic  letters  in  Fig.  5 
with  a  logical  one  response,  and  the  numbers  and  Greek 
letters  with  a  logical  zero  response.  These  characters 
were  chosen  from  a  group  of  50  tested  because  they  seemed 
to  be  the  most  difficult  for  the  machine  to  classify.  The 
input  vectors  are  shown  below  in  the  order  that  they  were 
presented  to  the  perceptron. 


si  -  0  0  1  0  0 
§5-00100 
8b  -  11100 

§3-11111 

§5*11100 

sj  *  0  0  1  10 


0  10  10  0  1 
0  0  1  0  0  0  0 
10  0  10  1  1 
0  0  0  1  0  0  0 
10  0  10  10 
0  10  10  11 


010111 
100001 
10  0  10  0 
110  10  0 
0  10  10  0 
1110  0  0 


1  1  1  0  0  0  1 
0  0  0  0  1  0  0 
10  11  10  0 
0  10  1110 
10  11  10  0 
1  0  0  0  0  1  0 
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■  1111110 
=  11110  10 
=  0111010 
=  111110  0 
=  1  0  0  0  1  1  0 
=  011  1010 
=  01  1  1000 
=  11110  10 
=  0111010 
=  0  0  1  0  0  0  0 
=  0111010 
=  111110  1 
=  111110  0 
=  111110  1 
*  1  0  0  0  1  0  1 
*10  10  110 
=  111110  0 
*111110  0 
sj  =  0  1  0  1  0  1  0 


0  0  0  1  1  1  0  0  1 
0  0  0  1  1  1  1  0  0 
0  0  0  1  0  0  1  1  1 
0  0  1  0  0  0  1  0  0 
0  0  1111111 
0  0  1  1  0  0  0  1  1 
1  0  0  0  0  1  0  0  0 
0  0  0  1  0  0  0  0  1 
0  0  1  1  0  0  0  1  1 
1  0  0  0  1  0  1  0  1 
0  0  0  0  1  1  0  0  0 
000001000 
1  0  0  0  0  1  0  0  0 
0  10  0  10  10  0 
0  1  0  0  0  1  0  0  0 
10  1111110 
0  1  0  0  0  1  0  0  0 
0  0  0  0  1  1  1  0  0 
1  0  1  0  0  1  0  0  0 


00001  0000 
0  0  0  1  0  1  1  1  0 
0  0  0  1  0  1  1  1  0 
0  1  0  0  0  1  0  0  0 
0  0  0  1  1  0  0  0  1 
0  0  0  1  0  1  1  1  0 
0  1  0  0  0  1  1  1  0 
00001  0000 
0  0  110  1111 
0  0  0  1  1  0  0  0  1 
0  0  10  1110  0 
1  0  0  0  1  1  1  1  1 
0  1  0  0  0  0  1  0  0 
10  10  0  10  10 
1  0  1  0  1  0  0  0  1 
0  1  0  0  0  0  1  0  0 
1  0  0  0  1  1  1  1  1 
0  0  0  0  1  1  1  1  1 
0  1  0  0  0  0  1  0  0 
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As  in  the  previous  problem,  the  weight  adjustments 
were  discontinued  after  the  perceptron  had  learned  to  rec¬ 
ognize  the  model  patterns,  and  noisy  input  vectors  were 
presented.  All  of  the  625  possible  vectors  with  one  noisy 
digit  and  600  of  the  7500  possible  input  vectors  with  two 
noisy  digits  were  shown  to  the  machine. 

Results  of  Problem  2.  Each  input  vector  had  to  be 
presented  to  the  perceptron  52  times  before  the  maohine 
learned  to  recognize  the  patterns. 

Of  the  625  input  vectors  with  one  digit  noisy,  92 
were  given  the  wrong  classification.  Thus  the  overall  rec¬ 
ognition  level  was  85$.  Table  3  is  a  listing  of  all  of  the 
errors  made. 

Only  75^  of  the  input  vectors  with  two  noisy  digits 
were  recognized.  There  were  149  mistakes  made  in  classi¬ 
fying  the  samples.  The  mistakes  are  tabulated  in  Table  4. 

Problem  Por  the  third  problem,  the  number  of  sen¬ 
sor  (S)  units  was  reduoed  to  nine.  This  reduction  was  made 
so  that  the  results  of  the  problem  could  be  compared  to  the 
results  obtained  from  a  simulation  of  the  Self-Organizing 
Binary  Logical  network  described  in  the  next  chapter.  Be¬ 
cause  50  association  units  were  used  for  this  problem  as 
well  as  the  first  two  problems,  a  qualitative  analysis  of 
the  effects  of  increasing  the  ratio  of  association  units 
to  sensor  units  can  also  be  made. 
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Character 

Number  of 
Mistakes 

Recognition 

Level 

Critical 

Digits 

A 

- 5 - 

80$ 

- 1 - 

- 5 

80* 

■UHU3Kl?JKi!JK£flkHH 

- E - 

- 8 

10W 

3 

7 

72* 

- B - 

- 2 - 

■IKUHHHBl 

4 

- o - 

- E - 

- ? - 

64* 

VWffBiilBHH 

- 5 - 

- 0 

G 

7 

72* 

1 

12 

— a - 

- 5 - 

4.  9 _ 

— o - 

2 

■UW^HI 

— i - 

1 

95*  : 

U  _ 

8 

68* 

■  TMl—M — — 

- 5 - 

2  .  i 

■iBfemHitariilH 

5 

&**  1 

nta uw||l|Ll| 

- § - j 

- 1 - 

■wiaB 

10 

60* 

- tjt - 

- 4 - 

84* 

^T712,  14,  13 _ 

- 3 

55* 

- X - 

- 5 

80* 

wuBiM^KUilli 

1 

mamismm 

- Z - 

- 5 - 

mmmiz jbb 

i 

- <3 - 

- 7 - 

72* 

— 

1,  5,  7,  16,  21,  22 
_25 _ 

Table  III 

Results  of  One-Digit  Noise  Tests  of  Problem  2 
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Vector 

Number 

Character 

Number  of 
Mistakes 

Recognition 

Level 

2  Critical 
Digits* 

1 

A 

1 

mmaasmm 

1 1 

2 

1 

20 

17* 

# 

all  except  10, 
12,  14,  21 

3 

B 

1 
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Table  IV 

Results  of  Two-Digit  Noise  Tests  of  Problem  2 


* 

The  digit  corresponding  to  the  vector  number  was 
always  noisy  in  these  tests,  so  these  digits  were  not 
included  in  this  column.  Thus,  vectors  particularly  sensi¬ 
tive  to  this  digit  were  almost  always  incorrectly  classified. 
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Because  there  are  very  few  letters  that  can  he  formed 
on  a  3  x  3  grid,  the  characters  x  and  +  were  used.  These 
symbols  have  only  one  element  of  their  input  vectors  in 
oommon.  The  patterns  and  their  input  vectors  are  shown 
below. 

H  * 

£  =  101010101  sT  =  oiomoio 

*x  «■+ 

After  the  perceptron  had  learned  to  recognize  these 
patterns,  the  weight  adjustments  were  stopped,  and  the  per¬ 
ceptron  was  asked  to  classify  all  of  the  18  possible  pat¬ 
terns  with  one  digit  perturbed. 

Results  of  Problem  2»  Six  different  random  topology 
(  [t] )  matrices  were  used  to  simulate  six  different  percep- 
trons.  Each  of  the  perceptrons  learned  to  correctly  iden¬ 
tify  the  x  and  the  +  after  seeing  each  vector  only  once. 
Pour  of  the  peroeptrons  classified  all  18  noisy  patterns 
oorrectly.  Each  of  the  other  two  perceptrons  incorreotly 
classified  one  of  the  18  veotors  with  one  digit  changed. 
Thus,  four  of  the  perceptrons  had  a  lOOjt  recognition  level, 
and  two  had  a  94 £  recognition  level. 

Disousslon  of  Problem  Results.  The  results  of  the 
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simulation  experiments  support  the  following  qualitative 
theories: 

1 .  An  increase  in  the  ratio  of  association  units  to 
sensor  units  tends  to  speed  up  the  learning  process  and  to 
inorease  the  noisy  pattern  recognition  capabilities  of  a 
simple  perceptron. 

2.  An  increase  in  the  number  of  olasses  of  patterns 
to  be  recognized  will  tend  to  slow  down  the  learning  pro¬ 
cess  and  decrease  the  noisy  pattern  recognition  capabili¬ 
ties  of  a  simple  perceptron. 

3.  The  pattern  recognition  capabilities  of  a  simple 
perceptron  are  functions  of  the  topology  matrix.  Thus, 
some  perceptrons  will  perform  better  than  others  on  a  given 
set  of  patterns. 

It  should  be  recalled  that  at  least  loggh  response 
units  are  required  to  completely  specify  the  responses  to 
k  pattern  classes.  Thus,  for  problem  2,  five  response 
units  would  be  required  to  function  properly  in  order  to 
differentiate  between  eaoh  of  the  25  pattern  classes. 
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III.  Statistical-Switching  Maohlnea 

Another  approach  to  the  design  of  self-organizing  ma¬ 
chines  is  to  use  standard  logical  devices  such  as  AND,  OR, 
and  NOT  gates  connected  together  hy  statistical  switches. 
Statistical  switches  hare  a  variable  probability  of  being 
closed  at  any  given  time.  Research  on  this  class  of  ma¬ 
chines  has  been  conducted  for  the  Air  Force  by  Melpar, 
Incorporated.  Several  devices  have  been  built  for  test 
purposes. 

Several  methods  for  connecting  the  gates  and  switohes 
have  been  investigated.  One  approach  is  to  uee  a  basic 
building  blook  called  an  Artron  (Ref  3),  a  two-input  one- 
output  network  that  is  oapable  of  learning  any  of  the  16 
switching  functions  of  two  variables.  The  Artrons  may  be 
connected  together  in  a  completely  random  network,  or  they 
may  be  conneoted  in  a  systematic  manner.  In  either  case 
the  networks  cannot  be  analyzed  except  on  a  statistical 
basis. 

The  Self-Organizing  Binary  Logical  Network  (SOBLN) 

can  be  studied  muoh  more  easily.  A  SOBLN  oan  learn  any  of 

2n 

the  2  Boolean  functions  of  n  variables,  which  is  not 
necessarily  true  of  Artron  networks.  Furthermore,  the  num¬ 
ber  of  statistical  switches  required  to  construct  a 
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completely  general  SOBLN  la  a  minimum  (Ref  3:97)* 

Sinoe  moat  atudiea  of  a  Self -Organizing  Binary  Logioal 
Network  can  be  applied  to  Artron  networks  (Ref  3*97),  the 
SOBLN  will  be  the  only  machine  discussed  in  detail  in  this 
chapter.  First,  the  operation  of  these  machines  will  be 
explained,  and  then  a  computer  simulation  of  a  SOBLN  will 
be  discussed. 

A  Self-Q -rganizinff  Binary  Logical  Network 

Number  of  Components  Required.  A  completely  general 

Self -Organizing  Binary  Logical  Network,  one  that  can  learn 
2n 

any  of  the  2  Boolean  functions,  must  have  as  many  AND 
gates  as  there  are  minterms  of  the  n  input  variables.  Thus, 
a  generalized  SOBLN  must  have  2n  AND  gates. 

As  in  the  peroeptron,  there  must  be  at  least  log2k 
outputs  to  completely  specify  k  pattern  classes.  For  eaoh 
output  there  must  be  one  OR  gate  and  2n  statistical 
switches  to  connect  the  AND  gates  to  the  OR  gates.  Thus,  a 
SOBLN  must  have  at  least  log2k  OR  gates  and  2n  loggk  sta¬ 
tistical  switches. 

Every  SOBLN  must  have  as  many  NOT  gates  as  there  are 
sensor  (input)  units.  A  SOBLN  with  n  sensor  units,  n  NOT 
gates,  2n  AND  gates,  2n  statistical  switches,  and  one  OR 
gate  is  shown  in  Fig.  b. 

Analysis.  For  each  set  of  inputs  corresponding  to  a 
pattern,  the  outputs  of  the  sensor  units  of  the  SOBLN  may 
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Pig.  6 

A  Self-Organizing  Binary  Logical  Network 
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be  arranged  in  the  form  of  a  binary  vector 


(12) 


where  s^  is  the  output  of  the  ith  sensor  unit.  As  in  the 
analysis  of  the  perceptrony  this  vector  will  be  oalled  the 
input  vector  and  will  often  be  written  in  its  transposed 
form 

S  =  |b .j  ®2  ® •  •  s ^ •  Sjjj  (13) 


The  output  of  each  sensor  unit  is  connected  to  an  inverter 
so  that  both  s^  and  its  logical  complement,  s^,  are  available. 

The  sensor  units  and  inverters  are  connected  to  the  AND 
gates  by  a  network  that  can  be  described  by  a  connection 
matrix 


C 


°11  c12*  *  *°1 j  *  *  *°1n 
C21  c22*  *  *°2j*  *  *°2n 

•  e  e  e 

e  e  e  e 

•  •  #  # 

Ci1  ci2*  *  *cij*  *  *°in 

e  •  e  e 


Cm1  °m2  *  *  *  °m J  *  *  *  cmn 


(14) 
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where  each  is  the  connection  of  the  jth  sensor  unit  or 
inverter  to  the  ith  AND  gate.  The  value  of  c^  may  be  zero 
or  one.  If  c^  is  one,  the  connection  is  made  directly  to 
the  sensor  unit.  If  is  zero,  the  connection  is  made  to 
the  inverter.  The  connection  matrix  may  be  arranged  in  the 
form  of  an  n-variable  truth  table  with  each  of  the  2n  rows 
corresponding  to  one  of  the  2n  minterms  or  oanonical  prod¬ 
ucts  of  the  n  inputs. 

Because  each  Ain)  gate  is  oonnected  in  the  manner  des¬ 
cribed  by  the  connection  matrix  of  Eq.  (H),  only  one  AND 
gate  will  have  a  logical  one  response  for  any  given  input 
vector.  All  other  AND  gate  responses  will  be  a  logical 
zero.  Thus,  the  response  of  the  ith  AND  gate  can  be  writ¬ 
ten  as 


ai 


i  if  sT  =  sA 
0  if  ST  »  c± 


(15) 


'  T 

where  a^  is  the  response  of  the  ith  AND  gate,  g  is  the  in¬ 
put  vector  and  I-8  the  ith  row  vector  of  the  connection 
matrix. 

Each  AND  gate  is  connected  to  the  output  OR  gate  by  a 
statistical  switch.  The  only  statistical  switch  that  can 
possible  have  a  logical  one  output  is  the  switch  that  is 
connected  to  the  AND  gate  that  had  a  logical  one  output. 
That  switch  will  have  a  logical  one  output  only  if  it  hap¬ 
pened  to  be  closed  at  the  time  that  the  AND  gate  response 


\ 
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occurred.  If  a  statistical  switch  has  a  logical  one  out¬ 
put  ,  the  OR  gate  will  have  a  logioal  one  output.  If  the 
state  of  a  switch  at  any  given  time  is  denoted  as 

0  if  the  ith  statistical  switch  is  open 

g±  =  S  (16) 

1  if  the  ith  statistical  switch  is  closed 

*  where  g^  is  the  state  of  the  ith  statistical  switch,  the 

response  of  the  SOBLH  to  an  input  vector  is 

t 

1  if  a^=1  and  g^=1 

R  =  S  (17) 

0  if  a^=0  and  g^-0 

Organization  Procedure.  The  method  for  organizing  a 
SOBLN  Involves  adjusting  the  closure  probabilities  of  the 
statistical  switches  after  each  input  vector  is  presented. 
Only  the  probability  of  the  switch  associated  with  the  AND 
gate  which  had  a  logical  one  output  is  adjusted  (Ref  3:16). 
The  probabilities  of  the  statistical  switches  are  initially 
set  to  be  0.3.  If  the  desired  response  is  a  logical  one, 
the  probability  may  be  increased  to  a  maximum  of  1.0.  If 
the  desired  response  for  the  input  vector  is  zero,  the 
probability  may  be  decreased  to  a  -inimum  of  zero. 

The  rules  for  adjusting  the  switch  probability  are 
(Ref  3:99): 

1.  If  the  statistical  switoh  was  closed,  a  reward 
increases  the  probability  and  a  punishment  decreases  the 
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closure  probability. 

2.  If  the  statistical  switch  was  open,  a  reward  de¬ 
creases  the  probability  and  a  punishment  increases  the 
closure  probability. 

These  rules  can  be  simplified  using  Boolean  algebra. 

Let 

■  response  of  the  ith  AND  gate  to  last  input  rector 
U  «  increase  probability  of  statistical  switch 
1)  =  decrease  probability  of  statistical  switch 
G  =  switch  was  closed 
(»  =  switch  was  open 

F  =  punish  switoh 
7  a  reward  switch 

Then 

U  =  A^gF  +  5p)  (18) 

and 

D  *  Ai(GF  +  SF)  (19) 

But  F  and  F,  the  punish  and  reward  variables,  can  be  writ¬ 
ten  in  terms  of  the  desired  response.  Let 

Rd  a  desired  response  is  1 
5d  >  desired  response  is  0 

Then 

F  -  (AjG)^  +  (I^)Rd  (20) 
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and 

7  ■  (A^G)R^  +  (XpF)R^  (21) 

Substituting  Eqs  (20)  and  (21)  for  F  and  7,  Eqe  (18)  and 
(19)  become 

U  *  A1{G[(AiG)Rd  +  (X^5F)Rdj  +  ^[(A^G)!?^  +  (A^GjR^J^  (22) 

D  =  A1|G[(A1G)5d  +  (X^)Rd]  +  ^[(AiG)Rd  +  (Xp)Ed]J  (23) 
Simplifying,  Eqe  (22)  and  (23)  can  be  written  as 

U  =  AdRd  (24) 

D  =  A±Sd  (25) 

Thus,  the  rules  can  be  reduced  to  the  following  statements: 

1 .  If  the  desired  response  is  a  logical  one ,  and  the 
ith  AHD  gate  response  is  one,  Increase  the  closure  proba¬ 
bility  of  the  ith  statistical  switch. 

2.  If  the  desired  response  is  zero,  and  the  1th  AND 
gate  response  is  one,  decrease  the  closure  probability  of 
the  1th  statistical  switch. 

For  striotly  combinatorial  pattern  recognition  prob¬ 
lems,  the  probabilities  can  be  changed  Immediately  to  zero 
or  one,  depending  upon  the  desired  response.  If  the  prob¬ 
lem  is  of  a  sequential  nature,  the  probability  should  be 
ohanged  in  small  increments. 
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Noisy  Pattern  Recognition.  It  can  be  seen  from  the 
following  statements  that  the  probability  of  recognizing  a 
noisy  pattern  correotly  is  0.5.  Only  the  probabilities  of 
the  statistical  switches  associated  with  the  model  input 
vectors  are  adjusted  during  the  training  process.  If  a 
noisy  pattern  is  presented,  the  AND  gate  which  has  a  logi¬ 
cal  one  response  is  connected  to  the  OR  gate  through  an 
untrained  statistical  switch.  Sinoe  the  untrained  statis¬ 
tical  switch  has  a  closure  probability  of  0.5»  the  recog¬ 
nition  level  will  approach  50#  when  many  noisy  input 
vectors  are  shown  to  the  machine  or  when  one  noisy  pattern 
is  repeatedly  given  to  the  SOBLN. 

A  partial  SOBLN  could  be  constructed  by  using  less 
than  2n  AND  gates  and  statistical  swltohes.  If  an  input 
veotor  that  was  not  equal  to  one  of  the  row  vectors  of  the 
connection  matrix  were  presented  to  the  partial  SOBLN  then 
none  of  the  AND  gates  would  have  a  logical  one  response. 
Because  none  of  the  AND  gates  would  have  a  logical  one 
response,  the  output  of  the  SOBLN  would  always  be  zero  for 
any  such  input  vector.  If  it  is  assumed  that  there  are 
equally  as  many  desired  responses  equal  to  one  as  there 
are  desired  zero  responses  for  a  large  set  of  input  vectors, 
then  the  probability  of  correctly  identifying  an  input 
vector  for  which  there  is  no  corresponding  AND  gate  ap¬ 
proaches  0.5.  Indeed,  it  appears  that  if  the  responses 
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for  all  of  the  possible  input  too tors  for  any  logical  net¬ 
work  are  not  specified  in  the  design,  the  probability  of 
correctly  responding  to  cm  unspecified  vector  will  approaoh 
0.5. 

Sequential  Pattern  Recognition.  A  job  that  the  SOBLN 
is  better  suited  for  is  the  recognition  of  patterns  in  se¬ 
quences  of  events.  Thus,  the  SOBLN  may  learn  to  anticipate 
the  next  number  of  a  series  of  numbers,  the  correct  turn 
for  a  mechanioal  rat  to  make  when  running  a  maze,  or  to 
play  a  competitive  game  with  a  biased  opponent.  This 
theory  is  supported  by  the  study  of  the  type  of  games  des¬ 
cribed  in  Ref  3. 

Computer  Simulation  of  a  SOBLN 

A  computer  prograa  to  simulate  a  Self -Organizing 
Binary  Logical  Network  was  written  by  the  author  for  an 
IBM  1620  digital  computer.  Because  over  33  million  AND 
gates  and  statistical  switches  would  be  required  for  a 
25-input  SOBLN,  the  first  two  problems  given  to  the  percep- 
tron  could  not  be  simulated.  The  number  of  sensor  units 
was  reduoed  to  nine  so  that  only  512  AND  gates  and  statis¬ 
tical  switches  were  neoessary.  As  in  the  perceptron  ex¬ 
periments,  only  one  output  unit  was  used. 

The  statistical  switch  probabilities  oould  be  varied 
from  0.50  to  either  0.00  or  0.99  in  six  steps  by  changing 
the  value  of  a  oounter.  The  counter  values  and  their 
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corresponding  probability  values  are  shown  in  the  table 
below. 


Counter 

Value 

0 

1 

2 

3 

4 

5 
b 

7 

8 
9 

10 

11 

12 


Probability  That  Statistical 
Swiich  is “Closed 

0.00 

0.01 

0.03 

0.06 

0.12 

0.25 

0.50 

0.75 

0.87 

0.93 

0.96 

0.98 

0.99 


Noisy  Pattern  Baenpni tion  Problem.  Problem  3  of  Chap¬ 
ter  II  was  also  presented  to  the  SOBLN  simulation  program. 
This  problem  required  the  SOBLN  to  learn  to  classify  the 
characters  x  and  +  with  a  logical  zero  and  a  logical  one 
respectively.  The  input  vectors  corresponding  to  these 
characters  are  repeated  below. 


si  =  101010101 

'-’X 

sT  =  010111010 


(26) 


After  the  SOBLN  had  learned  to  recognize  these  patterns, 
all  of  the  18  possible  patterns  with  one  noisy  digit  were 
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presented  to  the  machine.  These  same  18  patterns  were  shown 
to  the  SOBLN  23  times  for  a  total  of  414  trials. 

Results.  The  SOBLN  learned  to  correctly  identify  the 
model  input  rectors  after  seeing  eaoh  yeotor  eiz  times. 

The  maohine  failed  to  identify  correctly  202  of  the  414 
noisy  input  rectors.  Thus,  the  recognition  lerel  was 
approximately  51^.  Both  of  these  results  support  the  theo¬ 
retical  analysis  of  the  Self -Organ! zing  Binary  Logical 
Network.  It  is  interesting  to  note  that  Buroker  (Ref  2) 
obtained  similar  results  with  the  same  data  on  an  existing 
statistical-switching  machine. 
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IV.  Correlation  Machine e 

Another  approach  to  the  pattern  recognition  problem  is 
to  use  correlation  techniques.  These  techniques  are  dis¬ 
cussed  by  Highleyman  (Ref  6)  and  Fischler  (Ref  3). 

The  correlation  of  two  vectors  is  a  measure  of  the 
agreement  of  the  two  vectors.  The  degree  of  correlation 
between  two  n-component  binary  vectors  is  defined  here  to 
be  the  number  of  corresponding  components  of  the  two  vec¬ 
tors  that  agree.  Thus,  the  degree  of  correlation  of  two 
equal  vectors  is  r..  The  degree  of  correlation  between  two 
unequal  vectors  ie  n  minus  the  Hamming  distance  between  the 
two  veotors. 

The  assumption  haj  been  made  in  this  report  that  noisy 
patterns  should  be  classified  in  the  same  class  as  the 
model  pattern  that  is  the  closest  in  terms  of  Hamming  dis¬ 
tance  to  the  noisy  pattern.  A  correlation  pattern  recogni¬ 
tion  device  measures  the  degree  of  correlation  of  the  input 
vector  to  the  model  vectors,  determines  the  maximum  degree 
of  correlation,  and  then  classifies  the  input  vector  as  a 
member  of  the  same  class  as  the  model  vector  with  the  maxi¬ 
mum  degree  of  correlation  with  the  input  vector. 

Very  few  correlation  type  self-organizing  machines 
have  been  presented  in  the  literature.  The  Steele  Neuron 
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(Ref  1),  an  analog  device,  has  been  propoeed  for  use  in 
pattern  recognition  systems.  The  Self-Organizing  Pattern 
Recognition  Unit  (SOPFU),  which  is  the  only  device  that 
will  be  discussed  in  this  chapter,  is  proposed  by  the  au¬ 
thor. 

A  Self-fiTp*^  zi ng  Pattern  Recognition  Unit 

Humber  of  Component a  Required.  The  Self -Organizing 
Pattern  Recognition  Unit  shown  in  Pig.  7  must  have  n  sensor 
units,  n  HOT  boxes,  m  minterm  recognition  unite,  k  OR 
gates,  and  one  control  unit,  where  n  is  the  number  of  in¬ 
puts,  m  is  the  number  of  modal  input  vectors,  and  k  is  the 
number  of  pattern  clashes.  Each  minterm  recognition  unit 
(Pig.  8a)  contains  n  single-pole  double-throw  locking  re¬ 
lays,  one  2n-pole  single-throw  switoh,  one  summer,  and  one 
threshold  gate.  The  electronic  realization  of  the  minterm 
recognition  unit  (Pig.  9)  consists  of  n  bistable  multi¬ 
vibrators,  3n  AHD  gates,  one  summing  device,  and  one  thres¬ 
hold  gate.  The  control  unit  (Pig.  8b)  is  composed  of  a 
MAX  gate,  a  polarity  ohanger,  and  two  threshold  gates. 

Organization  Procedure.  Each  minterm  recognition  unit 
is  individually  trained  to  reoognize  one  model  input  vector. 
This  is  accomplished  by  closing  the  learning  control  switch 
in  the  minterm  recognition  unit  to  be  trained  and  present¬ 
ing  the  model  pattern  to  the  sensor  units.  Each  sensor 
unit  will  have  a  logioal  one  output  if  it  is  exoited  and  a 
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zero  output  othorwlso.  If  the  output  la  zero,  the  VOT  box 
will  hare  a  logioal  one  output.  Therefore,  either  the  sen¬ 
sor  unit  or  the  Inverter  will  always  have  a  logioal  one 
output,  and  the  corresponding  coll  of  the  locking  relay 
will  be  excited.  The  relay  will  switch  to  the  contact  with 
the  excited  coll  and  look  In  that  position. 

Beoause  all  relays  will  be  set  simultaneously,  the 
learning  process  for  each  model  vector  Is  completed  In  one 
step.  The  learning  oontrol  switch  should  then  be  opened. 
The  next  model  vector  can  then  be  taught  to  the  next  min- 
term  recognition  unit.  After  all  minterm  recognition  unite 
have  been  trained,  the  organization  procedure  is  complete. 
It  should  be  noted  that  all  of  the  information  necessary  to 
specify  a  model  veotor  is  oontained  in  the  position  of  the 
locked  relays  in  a  minterm  recognition  unit. 

Pattern  Recognition  Process.  When  a  pattern  is  pre¬ 
sented  to  the  SOPRU  after  the  organization  prooess  has  been 
completed,  the  relays  will  remain  In  the  same  position  be¬ 
cause  all  of  the  learning  control  switches  are  open. 
Therefore,  in  each  minterm  recognition  unit,  the  only  sum¬ 
ming  device  input  leads  that  will  have  a  signal  will  be 
those  which  are  oonneoted  through  the  relay  to  a  sensor 
unit  or  inverter  that  has  a  logioal  one  output.  Because 
the  positions  of  the  relays  were  determined  from  the  model 
input  vectors  during  the  training  process,  the  output  of 
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the  summing  device  will  be  a  direct  measurement  of  the  de¬ 
gree  of  correlation  between  the  input  rector  and  the  min- 
term  recognition  unit  model  vector. 

Ae  in  the  previous  chapter,  the  outputs  of  the  sensor 
units  can  be  arranged  in  the  form  of  a  column  vector 

S  = 


This  vector  will  be  referred  to  ae  the  input  vector. 

The  position  of  the  relays  of  all  of  the  mintera  rec¬ 
ognition  units  can  be  expressed  in  the  fora  of  a  connection 
matrix 


(27) 


C 


C11  °12*  *  *°1 *  *c1n 
e21  °22*  *  *c2j*  *  *°2n 

•  e  e  e 


ci2*  * •cij*  *  *cin 


(28) 


Cm1  cm2*  *  *cmj*  *  *c: 


mn 


where  c^  le  the  oonneotlon  of  the  summing  device  of  the 
ith  mintera  recognition  unit  to  the  Jth  sensor  unit  or  in¬ 
verter.  The  value  of  c^  will  be  zero  if  the  connection 
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la  made  to  the  Inverter  and  one  If  the  connection  la  made 
to  the  aenaor  unit.  Each  row  veotor  la  thua  identical  to 
the  correapondlng  model  vector. 

The  complement  of  a  binary  matrix  or  vector  ia  an 
array  in  which  eaoh  of  the  elementa  of  the  original  array 
i8  replaced  by  ita  complement.  Thua 


and 


°m2  *  *  *  °mn 


(29) 


(30) 


The  output  of  the  summing  device  of  eaoh  minterm  rec¬ 
ognition  unit  ia  the  degree  of  correlation  of  the  input 
vector  with  the  minterm  recognition  unit  model  vector.  The 
degrees  of  correlation  of  all  of  the  minterm  recognition 
units  can  be  arranged  in  a  column  vector 

(31) 
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where  each  is  the  degree  of  correlation  of  the  input 
rector  with  the  ith  mlnterm  recognition  unit  model  rector. 
This  rector  can  be  calculated  from 

5  -  C  S  +  S  5  (32) 

The  summing  derice  outputs  of  all  m  of  the  mlnterm 
units  are  the  inputs  for  the  MAX  gate  of  the  control  unit 
(Fig.  8b).  The  MAX  gate  has  as  its  output  the  maximum 
ralue  of  its  inputs.  A  simple  diode  OR  gate  will  produce 
an  output  that  is  the  maximum  ralue  of  its  inputs  and  can 
be  used  as  a  MAX  gate. 

The  output  of  the  MAX  gate  is  the  ralue  of  the  highest 
degree  of  correlation  of  the  input  rectors  to  any  of  the 
model  reotors.  This  can  be  expressed  as 

D  *  max  [djJ  (33) 

where  D  is  the  maximum  degree  of  correlation. 

The  algebraic  eign  of  D  is  then  ohanged  by  a  polarity 
ohanger.  It  le  possible  that  the  polarity  changing  oper¬ 
ation  could  be  aoeomplished  by  rererelng  two  leads. 

The  output  of  the  polarity  changer  is  connected  to  the 
inputs  of  the  threshold  gates  of  all  of  the  mlnterm  recog¬ 
nition  units.  The  output  of  the  summing  derlce  of  a  min- 
term  recognition  unit  is  also  an  input  for  that  unit's 
threshold  gate.  The  ralue  of  the  threshold  ralue  is  zero, 
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and  all  input  weights  are  one.  The  threshold  gate  will 
have  a  logical  one  output  if  the  output  of  the  summing 
devioe  is  equal  to  the  maximum  degree  of  correlation.  This 
can  be  written  as 


Mi  =  < 


1  if  d1  - 
0  if  d±  - 


D  »  0 
D  <  0 


(34) 


where  Hi  is  the  output  of  the  ith  minterm  recognition  unit. 
Because  1)  is  the  maximum  degree  of  correlation,  can 
never  be  greater  than  D,  but  several  minterm  recognition 
units  may  have  degrees  of  correlation  equal  to  the  maximum 
value.  Thus,  the  input  vectors  will  always  be  placed  in 
the  same  classifications  as  the  model  vectors  to  which  they 
have  the  highest  degree  of  correlation. 

An  OR  gate  is  used  to  combine  the  outputs  of  all  of 
the  minterm  recognition  units  associated  with  model  input 
vectors  that  are  to  be  placed  in  the  same  classification. 
Thus,  a  logical  one  OR  gate  response  indicates  that  the  in¬ 
put  vector  has  a  maximum  degree  of  correlation  with  one  of 
the  model  veotors  of  the  OR  gate's  class. 

Encoding  Networks.  The  response  is  in  a  decoded  form. 
It  can  be  encoded  by  using  a  coding  network  to  connect  the 
OR  gate  outputs  to  at  least  log2h  terminals,  where  k  is 
the  number  of  olasses. 

A  more  flexible  way  to  provide  a  ooded  output  would 
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be  to  connect  eaoh  ninterm  recognition  unit  to  all  of  the 
OR  gates  through  switches.  In  this  case  a  minimum  of  loggk 
OR  gates  is  required  instead  of  k  OR  gates.  The  switches 
can  he  opened  or  closed  during  the  training  process  so 
that  any  desired  set  of  OR  gate  responses  can  he  obtained 
for  the  output  of  any  minterm  recognition  unit.  Thus,  the 
output  can  he  put  in  coded  form. 

Error  Detecting*  It  is  possible  that  a  pattern  will 
he  equally  close  to  two  or  more  model  rectors.  In  this 
case,  more  than  one  minterm  recognition  unit  would  respond 
to  the  pattern.  If  these  minterm  recognition  units  are 
connected  to  different  OR  gates,  the  pattern  will  he  clas¬ 
sified  in  two  different  classes.  If  all  of  the  OR  gate 
outputs  are  connected  to  a  threshold  gate  with  a  threshold 
value  of  two  and  unity  input  weights,  the  threshold  gate 
will  give  an  error  signal  when  two  or  more  OR  gates  have  a 
logioal  one  output.  If  the  encoded  output  is  used,  the 
threshold  gate  must  he  connected  to  the  minterm  recognition 
unit  outputs. 

Another  type  of  error  occurs  when  a  pattern  is  clas¬ 
sified  that  does  not  have  a  high  degree  of  correlation  with 
any  of  the  model  veotore.  This  type  of  pattern  probably 
does  not  belong  in  any  of  the  classes  and  should  he  re¬ 
jected.  A  rejeot  signal  can  he  obtained  by  connecting  a 
threshold  gate  to  the  output  of  the  polarity  changer.  If 


54 


GE/EE/63-8 

-B(  the  negative  of  the  maximum  degree  of  correlation,  ex- 
oeede  some  negative  threshold  value  a  reject  signal  would 
appear  at  the  output  of  the  gate.  A  good  value  for  the 
magnitude  of  the  threshold  probably  is  the  average  of  the 
maximum  degrees  of  correlation  of  each  model  input  vector 
with  the  other  model  vectors.  This  value  was  used  during 
the  computer  simulation  experiments  to  be  discussed  later. 

Possible  Modifications.  Several  possible  modifica¬ 
tions  can  be  made  to  the  system.  The  relays  may  be  re¬ 
placed  by  electronic  oomponents  for  a  more  sophisticated 
device.  If  a  less  complioated  unit  is  desired,  the  relays 
may  be  replaced  by  toggle  swltohes  to  make  an  organizable 
network. 

An  electronic  version  of  a  minterm  recognition  unit 
is  shown  in  Fig.  9«  The  relays  have  been  replaced  by 
bistable  multivibrators  and  AMI)  gates.  Before  each  minterm 
recognition  unit  is  trained  to  recognize  a  model  vector, 
all  of  the  bistable  multivibrators  are  set  to  their  zero 
states  by  a  pulse  on  the  Reset  lead.  An  enabling  signal 
allows  the  AMD  gate  on  the  input  of  the  bistable  multivi¬ 
brator  to  have  a  logical  one  response  if  the  corresponding 
sensor  unit  is  excited  when  a  model  pattern  ie  presented. 
Thus,  a  multivibrator  will  switch  to  the  one  state  only  if 
the  sensor  unit  is  exoited. 

After  the  minterm  recognition  unit  has  been  organized, 
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the  enabling  signal  is  turned  off.  During  the  pattern 
recognition  process,  one  and  only  one  of  the  AND  gates  con¬ 
nected  to  the  outputs  of  the  ith  multivibrator  will  have  a 
logical  one  response  if  the  ith  term  of  the  input  veotor 
equals  the  ith  term  of  the  model  vector.  Otherwise,  the 
outputs  of  both  of  the  ith  AND  gates  are  zero.  These  AND 
gate  outputs  are  added  in  the  summing  device  to  determine 
the  degree  of  correlation.  The  mathematical  analysis  of 
the  pattern  recognition  process  for  the  electronic  version 
is  the  same  as  the  analysis  for  the  relay  version  of  the 
minterm  recognition  unit. 

The  pattern  recognition  capabilities  of  the  SOPRU  may 
possibly  be  Improved  by  using  some  of  the  techniques  dis¬ 
cussed  in  Chapter  II.  The  minterm  recognition  unit  input 
weights  could  be  adjusted  to  emphasize  the  important  input 
veotor  components.  This  would  have  to  be  done  with  the 
constraint  that  the  sum  of  the  weights  must  always  be  a 
constant  equal  to  n  so  that  the  model  vectors  will  still  be 
classified  correctly.  More  study  is  necessary  to  determine 
the  weight  adjustment  procedure. 

Computer  Simulation  of  £  SOPRU 

A  computer  program  (Appendix  C)  was  written  by  the 
author  to  simulate  a  25-input  SOPRU  with  25  minterm  recog¬ 
nition  units  so  that  the  noisy  pattern  recognition  capabil¬ 
ities  of  the  SOPRU  oould  be  investigated  experimentally. 
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The  number  of  classes  equaled  the  number  of  model  input 
vectors  used,  so  no  OR  gates  were  simulated.  The  first 
portion  of  the  program  automatically  determined  the  aver¬ 
age  of  the  maximum  degrees  of  correlation  of  each  model 
input  vector  with  the  other  model  vectors.  This  value  was 
used  as  a  threshold  to  determine  if  a  rejeot  message  should 
be  printed.  Only  one  problem  was  simulated. 

Problem.  The  same  data  that  was  used  in  Problem  2  of 
Chapter  II  was  used  in  this  problem  so  that  a  comparison  of 
the  results  could  be  made.  The  25  input  vectors  of  Eqs 
(11)  will  not  be  repeated  here. 

The  model  vectors  were  used  as  the  connection  matrix. 
Thus,  there  was  no  learning  process  necessary  for  this  prob¬ 
lem.  The  625  one-digit-noisy  input  vectors  were  presented 
to  the  SOFRTJ  to  see  how  each  was  classified.  Then  600 
vectors  with  two  noisy  digits  were  shown  to  the  SOPRU. 

Thus,  1223  samples  were  given  to  the  machine. 

Problem  Results.  The  average  value  of  the  maximum 
degrees  of  correlation  of  each  model  veotor  with  the  other 
model  vectors  was  calculated  by  the  computer  to  be  20. 

Hone  of  the  vectors  presented  to  the  SOPRU  had  maximum  de¬ 
grees  of  correlation  below  that  threshold  value,  so  no  re¬ 
ject  messages  were  printed. 

Twelve  of  the  623  vectors  with  one  noisy  digit  were 
placed  in  two  different  classifications.  Pifteen  of  the 
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600  veotors  with  two  noisy  digits  wsrs  also  given  dual  clas¬ 
sifications.  In  all  1225  cases  one  of  the  classes  in  which 
the  pattern  was  classified  was  the  correct  olass.  Thus, 
the  one-digit-noisy  patterns  were  classified  without  ambi¬ 
guity  for  slightly  more  than  98#  of  the  samples ,  and  the 
vectors  with  two  noisy  digits  were  given  singular  classi¬ 
fications  for  slightly  less  than  98#  of  the  samples.  Fig. 

10  shows  the  noisy  characters  that  were  given  dual  classi¬ 
fications.  If  these  patterns  are  compared  to  the  model 
patterns  of  Fig.  5*  it  can  be  seen  that  the  noisy  patterns 
are  equally  close  to  the  two  model  patterns  listed  under 
the  noisy  patterns. 
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V.  Refults  and  Conclusions 

Some  of  the  advantages  and  diaadvantages  of  the  three 
classes  of  self-organizing  machines  disouaaed  in  the  pre¬ 
vious  chapters  can  he  seen  by  comparing  the  number  of  com¬ 
ponents  required  and  the  results  of  the  computer  simulation 
problems.  These  comparisons  will  be  made,  some  conclusions 
will  be  drawn,  and  several  recommendations  for  further 
study  will  be  stated. 

Comparison  of  Results 

Humber  of  Pnita  Required.  Because  all  of  the  machines 
disoussed  require  essentially  the  same  number  of  input  and 
output  units,  no  comparison  of  these  numbers  needs  to  be 
made.  There  is  a  distinct  dlffsrenoe  in  the  number  of  in¬ 
termediate  logioal  devices  necessary,  however. 

The  perceptron  requires  a  maximum  of  2n  association 
units,  where  n  is  the  number  of  Inputs.  The  minimum  number 
is  a  function  of  the  number  of  model  vectors  the  peroeptron 
is  required  to  learn,  the  learning  speed  desired,  and  the 
desired  noisy  pattern  classification  ability.  In  general, 
this  minimum  number  oan  be  much  less  than  211. 

A  completely  general  Self -Organizing  Binary  Logioal 
Hetwork  must  have  at  least  2n  AHD  gates  and  2n  statistical 
swltohes.  If  all  of  the  model  vectors  that  the  machine 
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will  be  required  to  learn  are  known  In  advance,  fewer  AND 
gates  and  statistical  switohes  may  be  used,  but  the  maohine 
will  not  be  able  to  learn  all  of  the  possible  patterns. 

The  Self-Organizing  Pattern  Recognition  Unit  requires 
only  as  many  mlnterm  recognition  units  as  there  are  model 
input  rectors. 

Learning  Time.  The  table  below  shows  the  number  of 
times  the  machines  had  to  see  eaoh  model  input  rector  dur¬ 
ing  the  organization  processes. 


Problem  2 

Problem  3 

Learning  Time 

Learning  Time 

Perceptron 

32 

1 

SOBLN 

- 

6 

SOPRU 

1 

A  slight  modification  of  the  program  would  hare  allowed  the 
SOBLN  to  learn  the  model  rectors  after  seeing  each  rector 
only  once. 

Noisy  Pattern  Recognition  Levels.  The  abilities  of 
the  machines  to  reoognlze  the  noisy  patterns  of  Problems  2 
and  3  are  listed  in  the  table  below.  The  recognition  level 
is  the  fraotion  of  the  patterns  that  were  recognized  cor¬ 
rectly. 
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Problem  2  Recognition 

Levels 

Problem  3  Recognition 
Levels 

One  Digit 
Noisy 

Two  Digits 
Noisy 

One  Digit  Noisy 

Perceptron 

85* 

75* 

100*  for  4  perceptrons 
94*  for  2  perceptrons 

SOBLN 

— 

— 

51* 

SOPRTJ 

98* 

98* 

— 

It  should  be  recalled  that  the  SOPRTJ  was  required  to 
separate  the  vectors  into  25  different  classes  for  problem 
2,  but  the  perceptron  had  to  distinguish  between  two  class¬ 
es  only.  The  perceptron,  however,  was  handicapped  because 
it  had  a  small  ratio  of  association  unite  to  sensor  units, 
and  it  only  had  one  response  unit. 
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Conclusions 

The  conclusions,  based  on  this  investigation  are  as 
follows : 

1 .  Self-organizing  machines  which  use  adjustable- 
weight  threshold  logic,  such  as  perceptrons  with  high 
ratios  of  association  units  to  sensor  units,  are  well  suit¬ 
ed  to  solve  pattern  recognition  problems  because  of  their 
ability  to  classify  most  noisy  patterns  correctly.  Their 
major  disadvantage  is  the  relatively  large  number  of  times 
the  model  input  vectors  must  be  presented. 

2.  Statistical-switching  machines,  such  as  the  Self- 
Organizing  Binary  Logioal  Network,  cannot  be  used  to  rec¬ 
ognize  noisy  patterns.  Furthermore,  they  are  not  practical 
for  problems  requiring  large  numbers  of  inputs  (sensor 
units)  because  they  require  2n  AND  gates  and  2n  statistical 
switches.  The  major  advantage  of  the  SOBLN  is  the  rela¬ 
tively  few  number  of  tines  the  model  input  vector  must  be 
shown  to  the  machine. 

3.  Machines  using  correlation  techniques,  such  as  the 
self-organizing  pattern  recognition  unit,  combine  some  of 
the  advantages  of  the  perceptron  and  the  SOBLN.  They  have 
a  high  degree  of  ability  in  noisy  pattern  recognition. 

They  require  relatively  few  components.  Furthermore,  the 
SOFRU  must  see  eaoh  model  vector  only  once. 
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Recommendations  for  Further  Study 

Several  topics  are  reoommended  for  further  investi¬ 
gation: 

1 .  It  is  possible  that  the  set  of  weights  determined 
by  the  oi  -error  correction  procedure  in  the  perceptron  is 
not  the  optimum  feasible  solution  for  noisy  pattern  rec¬ 
ognition.  It  may  be  possible  to  apply  some  of  the  tech¬ 
niques  of  linear  programming  to  determine  an  optimum 
feasible  solution  that  would  maximize  the  distance  of  each 
model  vector  from  the  separating  hyperplane.  This  may 
correspond  to  maximizing  slack  variables. 

2.  The  weights  on  the  inputs  of  the  SOPRU  all  have  a 
value  of  one.  An  adjustment  procedure  could  be  derived  as 
suggested  in  Chapter  IV  that  would  improve  the  noisy  pattern 
recognition  capabilities  of  the  SOPRU  by  ohanglng  the  value 
of  these  weights. 

3.  An  organizable  pattern  recognition  unit  could  be 
constructed  using  the  SOPRU  recognition  techniques.  The 
relays  could  be  replaoed  by  inexpensive  toggle  switches. 

The  summing  devices  oould  be  built  using  Kirehhoff  adders. 
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Appendix  A 

Computer  Simulation  Program  for  a  Simple  Perceptron 

A  computer  program  to  simulate  a  simple  perceptron  on 
an  IBM  1620  digital  computer  is  shown  on  page  71 .  A  flow 
chart  for  this  program  is  shown  in  Pig.  A-1 .  The  number  of 
sensor  (S)  units  for  the  peroeptron  simulated  is  twenty- 
five,  and  the  number  of  association  (A)  units  is  fifty. 

Only  one  response  (R)  unit  is  used. 

The  outputs  of  the  A  units  can  be  stored  for  a  max¬ 
imum  of  twenty-five  different  input  vectors.  Thus,  during 
the  organization  process,  the  outputs  of  the  A  units  do  not 
have  to  be  recalculated  after  all  of  the  input  veotors  have 
been  presented  to  the  oomputer.  After  the  response  for 
each  input  vector  has  been  calculated,  the  computer  will 
type  out  the  number  of  the  input  vector,  the  actual  re¬ 
sponse,  and  the  desired  response.  If  console  switch  3  is 
on,  the  weights  will  be  adjusted  using  the  <x-error  correc¬ 
tion  procedure. 

The  program  was  written  in  the  APIT  Fortran  prooessing 
language.  APIT  Fortran  allows  a  free-style  input  format  on 
the  data  cards.  Thus,  the  input  data  may  be  punched  on 
cards  in  any  format  with  spaces  or  oommas  between  each  num¬ 
ber.  A  memory  capacity  of  40,000  decimal  digits  is  required 
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to  contain  the  program  and  the  data. 

Several  simple  modifications  of  this  program  can  be 
made  to  adapt  it  to  special  problems.  The  final  value  of 
"DO"  statements  31 ,  5,  and  9  can  be  changed  to  any  number 
less  than  twenty-five  to  change  the  number  of  sensor  (in¬ 
put)  units.  If  the  number  is  made  higher  than  twenty-five, 
the  appropriate  dimension  statements  must  be  changed.  Sim¬ 
ilarly,  the  final  values  of  "DO"  statements  11,  12,  32,  13, 
and  19  can  be  changed  to  vary  the  number  of  A  units.  The 
dimension  statements  must  also  be  changed  if  the  number  of 
A  units  desired  is  more  than  fifty. 

A  table  of  console  switch  settings  with  their  functions 
is  given  below. 


OH 

OFF 

SW#1 

Read  new  input 
vectors 

Do  not  read  input 
vectors 

SW#2 

Calculate  A  unit 

Use  stored  A  unit 

response 

response 

SW#3 

Adjust  weights  on 

R  unit  inputs 

Do  not  adjust  weights 
on  R  unit  inputs 
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Pig.  A-1 


Plow  Diagram  for  Peroeptron  Simulation  Program 


Compute  Weighted  Sum  of  Inputs  to  "A"  Unit 
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Pig.  A-1 


Plow  Diagram  for  Perceptron  Simulation  Program 


Print  Input  Vector  Number,  Actual 
Response,  Desired  Response 


Adjust  Weights? 


Have  All  Weights  Been  Adjusted? 


Yes 
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Computer  Program  for  a  Simple  Peroeptron 


C  PERCEPTRON  PROGRAM 

DIMENSION  IPETA( 50 ) , IW( 1 0 ) . IT( 25 , 50 ) , IS ( 25 ) 
DIMENSION  IRD(25),IA(25,50),IP(50) 


1  READ,KPETAfM 


11  DO  2  1*1,50 

2  READ ,  IW  ( 1 ) 

12  DO  3  1*1 ,50 

3  READ,IPETA( 1 ) 

31  DO  4  1*1,25 

32  DO  4  J*1 ,50 

4  READ,IT(l, J) 

41  PRINT  26 
DO  22  K-1 ,M 
IP (SENSE  SWITCH  1)5,7 

5  DO  6  J*1 ,25 

6  READ,IS(J) 

READ,IRD(E) 

7  IQ*0 

13  DO  17  1*1,50 
IF( SENSE  SWITCH  2)8,15 

8  IP(I)*0 

9  DO  10  J*1 ,25 

10  IP(l)*1P(1)+1T(1,J)*1S(J) 

15  IA(KI)*0 

IP(IP(l)-IPETA(l))17,16,1 

16  IA(K,I)*1 

17  IQ-IQ+IW(I)*IA(K,I) 

IR-0 

IP(IQ-KPETA)42, 18, 18 

18  IR*1 


Read  R  unit  threshold  value 
ft  number  of  input  rectors 

Read  aeoond  stage  initial 
weights 

Read  A  unit  threshold  value 


Read  first  stage  topology 
matrix 


Print  column  heading 
Set  rector  number  counter 
Bypass  input  rector  read 

Read  input  rector 

Read  desired  response 


Compute  weighted  sum  of  A 
unit  inputs 


Determine  A  unit  response 


Compute  weighted  sum  of 
R  unit 


Determine  R  unit  response 
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42  PRINT  25,K,IR,IRD(K)  Print  results 

IP (SENSE  SWITCH  3)19,22  Bypass  weight  adjustment 

20  IW(I)=IW(I)+IA(K,I)*(IRD(K)-IR)  «J^5J.8e°0nd  8tag® 
22  CONTINUE  weignts 

GO  TO  41 

25  P0RMAT(/I3,I2,I2) 

26  FORMAT (//8H  SN  R  RD) 

END 


Symbol  Table 

IA(K,I)  =  response  of  ith  A  unit  to  Kth  input  vector 
IFETA(I)  =  threshold  value  of  ith  A  unit  threshold  gate 
IP(I)  *  weighted  sum  of  inputs  to  ith  A  unit 
IQ  a  weighted  sum  of  inputs  to  R  unit 
IR  =  response  of  R  unit 

IRD(K)  »  desired  response  of  R  unit  to  Kth  input  vector 

IS(J)  *  value  of  jth  bit  of  binary  input  (sensor)  vector 

IT(I,J)  a  weight  value  of  input  to  the  ith  A  unit  from  the 
jth  S  unit 

IW(I)  a  weight  value  of  input  to  R  unit  from  the  ith  A  unit 
I  a  row  subscript 
J  a  column  subscript 
K  a  input  vector  number 

KFETA  a  threshold  value  of  R  unit  threshold  gate 
M  a  number  of  input  vectors 
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Appendix  B 

Computer  Simulation  Program  for  a  Self- 
Organizing  Binary  Logioal  Network 

The  computer  program  shown  on  page  79  will  simulate  a 

nine-input  Self-Organizing  Binary  Logical  Network.  For  a 

completely  general  pattern  recognition  device,  a  minimum  of 

one  statistical  switch  and  one  AND  gate  is  required  for 

q 

each  mlnterm;  therefore,  27  statistical  switches  and  AND 
gates  were  simulated. 

The  program  was  written  in  the  Symbolic  Programming 
System  (SPS)  language  for  an  IBM  1620  computer  with  indi¬ 
rect  addressing.  SPS  was  used  because  of  the  difficulty 
of  generating  a  random  number  and  because  of  the  large  mem¬ 
ory  capacity  required  when  FORTRAN  is  used  for  a  simulation 
program  of  this  sort. 

The  program  works  in  the  following  manner.  The  first 
stage  connection  matrix  consisting  of  all  of  the  binary 
numbers  from  0  to  311  is  read  into  memory.  Each  binary 
number  corresponds  to  the  mlnterm  associated  with  one  of 
the  AND  gates.  The  model  input  rectors  with  their  desired 
responses  are  read  into  memory  in  binary  form.  Each  input 
rector  is  comparsd  to  eaoh  binary  number  of  the  connection 
matrix.  The  binary  connection  rector  that  is  equal  to  the 
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input  yeotor  corresponds  to  the  AND  gate  whioh  will  have  a 
logical  one  output.  The  outputs  of  all  other  And  gates  are 
zero.  A  random  number  is  then  generated  using  Lehmer's 
method  (Ref  12:17).  The  random  number  is  compared  to  a 
threshold  value  corresponding  to  the  counter  value  of  the 
statistical  switch  associated  with  the  AND  gate  which  had  a 
logical  one  output.  If  the  random  number  is  less  than  the 
threshold  value,  a  logical  one  is  the  output  of  the  OR  gate. 
Otherwise,  the  output  is  a  logical  zero.  During  the  train¬ 
ing  period,  the  statistical  switch  counter  is  increased  to 
a  maximum  of  twelve  if  the  desired  OR  gate  output  is  a  log¬ 
ical  one,  and  the  counter  is  decreased  to  a  minimum  of  zero 
if  the  desired  output  is  zero.  Thus,  the  training  process 
is  complete  after  the  response  for  each  input  vector  has 
been  calculated  six  times.  A  flow  ohart  for  the  program  is 
shown  in  Fig.  B-1 . 

The  input  data  must  be  punched  on  cards  using  a  spe¬ 
cial  format.  There  must  be  a  flag  over  the  left-hand  digit 
of  each  number,  and  there  must  be  no  spaces  between  digits. 
The  initial  random  number  must  be  punched  in  the  first 
eight  oolumns  of  the  card.  Eight  9-digit  binary  numbers 
must  be  punehed  on  each  of  64  cards  for  the  connection  ma¬ 
trix.  Each  input  vector  must  be  punohed  in  the  first  nine 
columns  of  a  card.  All  of  the  desired  responses  must  be 
punohed  in  the  first  columns  of  one  card  with  a  zero  after 
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each  desired  response. 

During  the  training  period,  switch  1  may  be  turned  off 
to  speed  up  the  training  operation  by  bypaaeing  the  first 
stage  logio  after  the  AND  gate  responses  for  each  model  in¬ 
put  vector  hare  been  determined.  Switoh  2  oontrols  the  re¬ 
ward  and  punishment  of  the  statistical  switches  and  should 
be  turned  off  when  training  is  completed.  Switch  3  allows 
new  input  reotors  to  be  presented  to  the  maohlne.  A  table 
of  the  oonsole  switch  settings  is  shown  below. 

STITCH  01  OFF 


SW#1 

Calculate  AND 

By pa s 8  AND  gate 

gate  responses 

response  calculation 

SW#2 

Adjust  statistical- 

Bypass  probability 

switch  probability 

adjustment 

SW#3 

Bead  new  input 

Do  not  read  new 

reotors 

input  rectors 

i 
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Pig.  B-1 


Initialize  Input  Vector  Counter  and  Compare  Addressee 
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Pig.  B-1 


Plow  Chart  for  a  Computer  Simulation  of  a 
Self-Organizing  Binary  Logical  Hetwork 


* 


Pind  Counter  Value  for  Statietioal  Switch  Which 
Connects  AID  Gate  Which  Pired  to  Output  OH  Gate 


Determine  Threshold  Value  Corresponding  to  Counter  Value 


m 
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Pig.  B-1 


Have  Response s  to  M  Input  Vectors  \w  S'  "n 
Been  Calculated?  J“-(  LADD2 


Read  Hew  Input  Vectors? 
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Self -Organizing  Binary  Logioal  Network  Program 


4 


7 


START 

RICD 

BN-7 

TPM 

N,0, 10 

TPM 

ADDR,CIJ-8 

BCN 

AM 

N,1,10 

RNCD 

-ADDB 

AM 

ADDB, 72 

CM 

N,64,10 

BL 

BCN 

TPM 

N,0,10 

TPM 

ADDR,KI-1 

BO 

AM 

N,1,10 

RNCD 

-ADDB 

AM 

ADDB, 64 

CM 

N, 16, 10 

BL 

BO 

LSET1 

TPM 

L,0,10 

TPM 

RES+6 , SLI-8 

LADD1 

AM 

L,1,10 

RES 

RNCD 

SLI-8 

AM 

RES+6,9 

C 

L,2 

BL 

LADD1 

RNCD 

IBDL 

LSET2 

TPM 

L,0, 10 

TPM 

COMP+1 1 ,SLI-9 

LADD  2 

AM 

L,1,10 

MM 

L,512,9 

TP 

ADDB, 99 

AM 

ADDB,BI-512 

BNC1 

STA0E2 

TPM 

C0MP+6,CIJ 

AM 

COMP+1 1,9 

TPM 

1,0,9 

IADD2 

AM 

1,1,9 

COMP 

C 

Cl J , SLI 

BZ 

BSET1 

Bead  random  number 

Bead  number  of  input  vectors 


Bead  lat  stage  connection 
matrix 


Bead  statistical-switch 
counter  initial  values 


Bead  input  veotors 


Bead  desired  response 


Beset  counter  &  addresses 


Calculate  address  of  re¬ 
sponse  for  1st  AND  gate 


Bypass  to  stage  2 


Compare  input  veotor  to  minterm 
of  ith  AND  gate 
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RSETO 

TDM 

-ADDR,0 

AM 

ADDR,  1 

AM 

COMP+6,9 

CM 

1,512,9 

BL 

IADD2 

B 

STAGE2 

RSET1 

TDM 

-ADDR,  1 

B 

RSETO+12 

STAGE2 

MM 

L,512,9 

TF 

ADDR, 99 

AM 

ADDR,RI-512 

TFM 

1,0,9 

IADD7 

AM 

1,1,9 

DIG 

BD 

RAND, -ADDR 

AM 

ADDR, 1 

B 

IADD7 

RAND 

MM 

RN,23,10 

SF 

92 

S 

99,91 

TF 

BN, 99 

MM 

o 

T“ 

•* 

CM 

M 

AM 

99,KI-2 

TF 

ADDR1,99 

FRESH 

MM 

-ADDR1 ,8,9 

AM 

99>THRES 

C 

BN, -99 

BH 

RO 

TDM 

R,  1 

B 

PRINT 

RO 

TDM 

R,0 

PRINT 

RCTY 

BD 

TYP,L-1 

WNTY 

L 

B 

SPACE 

TIP 

WNTY 

L-1 

Set  response  ■  0 
Modify  addresses 

Checked  all  minterms? 

Set  response  =  1 

Find  which  AND  gate  fired 

Generate  random  number 

Find  statistical  switch 
counter  address 

Find  address  of  threshold 

Compare  random  number  to 
threshold 

Set  response  =  1 

Set  response  ■  0 

Print  input  ye c tor 
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SPACE 


ADDK1 


SUBK1 

LCK1 


IHRES 


I 

HI 


SPTY 

WNTY 

SPTY 

R 

Print  actual  response 

HM 

I»»2,9 

TP 

ADDR2 , 99 

AH 

TD 

ADDR2,IHDL-1 
-ADDR2 , 400 

Print  desired  response 

SM 

ADDR2  ,  1 

WETY 

-ADDR2 

BHC2 

LCK1 

Bypass  counter  adjustment 

BD 

ADDK1 ,-ADDR2 

Branch  if  desired  output  =  1 

BD 

STJBK1  ,-ADDR1 

Branch  if  counter  value  =  0 

B 

LCK1 

CM 

BEL 

-ADDR1 ,12,10 
LCK1 

Branch  if  counter  value  =  1 2 

AM 

B 

-ADDR1 ,1,10 
LCK1 

Add  1  to  counter  value 

SM 

-ADDR1 ,1,10 

Subtract  1  from  oounter  value 

C 

L»  2 

Have  all  responses  been 

BL 

LADD2 

determined? 

BC3 

LSET1 

Read  new  input  veotors 

B 

LSET2 

Do  not  read  new  input  vectors 

DC 

8,00000000 

DC 

8,01000000 

DC 

8,03000000 

DC 

8,06000000 

DC 

8,12000000 

DC 

DC 

DC 

8,23000000 

8,30000000 

8,75000000 

Statistical  switch  threshold 
values 

DC 

8,87000000 

DC 

8,93000000 

DC 

8,96000000 

DC 

8,98000000 

DC 

8,99000000 

DS 

2 

Card  counter 

DS 

2 

Input  veotor  subscript 

DC 

1.® 

DS 

3 

Row  subsoript 

DSS 

5120 

Response  of  1th  AED  gate 
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ADDR 

DS 

5 

Address  storage  ' 
for  R< 

ADDR1 

DS 

5 

Address  storage 
for 

For 

*  indireot 

addressing 

ADDR2 

DS 

5 

Address  storage 
for  RD^ 

R 

DS 

2 

Or  gate  response 

DC 

1,© 

RH 

DS 

8 

Random  Humber 

CIJ 

DSB 

9,512 

Minterm  of  ith  AND  gate 

KI 

DSB 

2,512 

in  statistical  switch  counter 

SII 

DSB 

9,10 

1th  input  vector 

IRDL 

DSS 

20 

Desired  response 
vector 

of  1th  input 

DERD 

START 
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Appendix  C 

Computer  Simulation  Program  for  a  Self- 
Organizing  Pattern  Recognition  Unit 

The  computer  program  for  the  simulation  of  a  Self- 
Organizing  Pattern  Recognition  Unit  (SOPRU)  was  written  in 
APIT  FORTRAN  for  an  IBM  1620  computer.  The  program  will 
simulate  a  25  input  SOPRU  with  25  minterm  recognition  units. 
Since  a  minimum  of  one  minterm  recognition  unit  is  required 
for  each  olass  of  patterns  to  he  recognized,  the  SOPRU  sim¬ 
ulated  will  separate  a  maximum  of  25  different  pattern 
groups. 

The  first  step  in  the  program  is  to  read  into  memory 
the  model  input  vectors.  If  console  switoh  1  is  on,  a 
threshold  value  is  then  determined  by  finding  the  maximum 
degree  of  correlation  of  each  model  veotor  with  the  other 
model  vectors  and  then  finding  the  average  of  these  max¬ 
imum  correlation  values.  If  the  threshold  value  is  already 
known,  the  threshold  value  oaloulation  can  be  bypassed  by 
turning  console  switoh  1  off  and  typing  in  the  threshold 
value . 

Twenty-five  input  vectors  representing  patterns  to  be 
recognized  are  then  read  into  memory.  Eaoh  digit  of  a 
binary  input  vector  is  then  compared  to  the  corresponding 
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digit  of  a  model  vector.  If  the  digits  are  equal,  a  one 
is  added  to  an  accumulator.  After  all  23  digits  have  been 
checked,  the  number  in  the  accumulator,  rhich  is  the  degree 
of  correlation,  is  oompared  to  previously  determined  degrees 
of  correlation  of  the  input  vector  to  the  other  model  vec¬ 
tors  to  see  which  is  a  maximum.  The  maximum  value  of 
these  degrees  of  correlation  is  then  compared  to  the  de¬ 
gree  of  correlation  of  the  input  vector  with  each  model 
vector.  Each  model  vector  with  a  degree  of  correlation  to 
the  input  vector  equal  to  the  maximum  degree  of  correlation 
corresponds  to  a  minterm  recognition  unit  with  an  output 
equal  to  a  logical  one. 

If  the  input  vector  was  improperly  classified,  the 
input  vector  number,  the  number  of  the  minterm  recognition 
unit  with  a  logical  one  output,  and  the  maximum  degree  of 
correlation  are  typed.  If  the  maximum  correlation  is  less 
than  the  previously  determined  threshold,  the  word  (Reject" 
is  typed.  After  b23  patterns  have  been  classified,  the 
program  stops. 

A  flow  chart  is  shown  in  Fig.  C-1 ,  followed  by  the 
program. 
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Pig.  C-1 


Flow  Chart  for  a  Computer  Simulation  of  a 
Self -Organizing  Pattern  Recognition  Unit 


Read  Model  Input  Vectors 


Calculate  or  Aocept  Threshold  Correlation  Value 


Print  Threshold  Correlation  Value 


,  +  , 

Read  25  Input  Vectors 

T  ' 

Determine  Degrees  of  Correlation  of 
Input  Veotor  to  Model  Vectors 


r  ■-  . 1 

Determine  Maximum  D< 

i _ , 

tgree  of  Correlation 

Determine  Which  Mint< 
Have  Output  =  Logic a] 
- : — , - - 

irm  Recognition  Units 

L  1 

1 - 1 
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Plow  Chart  for  a  Computer  Simulation  of  a 
Self -Organizing  Pattern  Recognition  Unit 


Print  Input  Vector  Group  Number,  Input 
Vector  Number,  Number  of  Minterm  Recog' 
nition  Unit  Which  had  Output  =  1 ,  and 
Maximum  Correlation  Value 
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Computer  Program  for  Self -Organizing 
Pattern  Recognition  Unit 


C  SOPRU  2 

DIMENSION  M0D(25,25),IPRES(25) 
DIMENSION  ISUM(25),IHPUT(25,25) 


DO  1  L=1 ,25 

DO  1  1=1,25 

1  READ  25,MOD(L,I) 

IP( SENSE  SWITCH  1)22,23 


Read  model  input 
vectors 

Bypass  minimum  correla^ 
tion  calculation 


22  IFETA=0 

DO  6  M=1,25 
IFRES(M)»0 
DO  5  L-1,25 
IF(L-M)21 ,5,21 
21  ISUM(L)=0 
DO  3  1=1,25 

IF(M0D(M,I)-M0D(L,I))3,2,3  Caloulate  minimum 

2  ISUM(L)»ISUM(L)+1  correlation  value 

3  CONTINUE 

IF(ISUM(L)-IFRES(M))5,5,4 

4  IFRES(M)=ISUM(L) 

5  CONTINUE 

6  IFETA=IFETA+IFRES ( M ) 

IFETA-IFETA/25 

GO  TO  7 


23  ACCEPT, IFETA 

7  PRINT  26, IFETA 

DO  18  M=1,25 

DO  8  K»1,25 
DO  8  1=1,25 

8  READ  25,INPUT(K,I) 


Read  minimum 
correlation  value 

Print  minimum 
correlation  value 

Set  group  counter 


Read  25  input  vectors 


Set  input  vector 
oounter 


DO  18  K«1,25 
JSUM-0 
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DO  12  L=1 ,25 


Set  minterm  counter 


ISUM(L)»0 
DO  1=1,25 

IF(lNPUT(K,l)-MOD(L,I) )10,9»  10 

9  ISUM(l)=ISUM(L)+1 

10  CONTINUE 


Determine  degree  of 
correlation 


IF(ISUM(L)-JSUM)  12, 12,11 

11  JSUM»ISUM(L) 

12  CONTINUE 


Determine  maximum 
correlation  value 


DO  18  L=1 .25 

IF ( ISUM( L ) - JSUM) 1 8 , 1 3 , 1 3 

13  IF(L-K)14,15,H 

14  FEINT  27,M,K,L,ISUM(L) 


Determine  which  minterm 
recognition  units  fired 

If  vector  is  not  clas¬ 
sified  correctly,  print 
results 


15 

16 
18 


IF ( JSUM-IFETA ) 1 6 , 1 8 , 1 8 
PRINT  28,M,K,L,ISUM(L) 
CONTINUE 


If  correlation  value  is 
helow  minimum  correla¬ 
tion  value,  print  "Reject" 


STOP 

25  FORMAT ( 150, 30X) 

26  F0RMAT(/20HMINIMUM  CORRELATION* ,  13  ) 

27  F0RMAT(/I3,I4,I4,I6) 

28  F0RMAT(/6HRBJECT,I4,I4,I4,I4,) 

END 
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Symbol  Table 


I  =  row  subscript 

I PET A  a  threshold  (minimum  correlation)  value 

IFHES(M)  «  degree  of  correlation  of  mth  model  vector 
with  other  model  vectors 

IHPUT(K,I)  «  ith  digit  of  kth  input  vector 

ISTJM(L)  ■  degree  of  correlation  of  1th  model  veotor  with 
input  vector 

JSUM  a  maximum  degree  of  correlation 
K  a  input  vector  number 

L  a  minterm  recognition  unit  (model  vector)  number 

U  a  input  vector  group  number 

MOD(L,I)  a  ith  digit  of  1th  model  vector 
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